Wavelet transformation device and method, wavelet inverse transformation device and method, program, and recording medium for performing wavelet transformation at a plurality of division levels

ABSTRACT

A wavelet transformation device for performing wavelet transformation at a plurality of division levels as to image signals. A horizontal filtering unit subjects the image signals to horizontal direction lowband analysis filtering and highband analysis filtering. Buffers which are independent for each of the division levels, hold frequency components, which are generated as the results of the horizontal direction analysis filtering by the horizontal filtering unit, for each of the division levels. Interleaving devices comprising at least one interleaving unit interleaving brightness components and color difference components of the image signals and interleaving highband components and lowband components of buffered filter results.

CROSS REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese PatentApplication No. 2006-193669 filed in the Japanese Patent Office on Jul.14, 2006, the entire contents of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a wavelet transformation device andmethod, a wavelet inverse transformation device and method, a program,and a recording medium, and more particularly relates to a wavelettransformation device and method, a wavelet inverse transformationdevice and method, a program, and a recording medium, enablinghigh-speed wavelet transformation to be performed by exchanging datawith internal memory.

2. Description of the Related Art

An image compression method representative of known methods is the JPEG(Joint Photographic Experts Group) method that has been standardized bythe ISO (International Standards Organization). This is known to provideexcellent coded images and decoded images in the event that DCT(Discrete Cosine Transform) is used and a relatively great number ofbits are appropriated. However, reducing coding bits beyond a certainlevel results in marked block distortion characteristic of DCT, anddeterioration can be subjectively observed.

On the other hand, in recent years, there has been much researchperformed on a method wherein images are divided into multiple bands(sub-bands) with filters called filter banks, wherein high-pass filtersand low-pass filters are combined, and coding is performed for eachband. Particularly, wavelet transformation coding is viewed as a newtechnology which is a likely candidate to replace DCT, since it does nothave the problem that DCT has, i.e., marked block distortion at highcompression rates.

International standardization of JPEG 2000 was completed January 2001.JPEG 2000 combines wavelet transformation and high-efficiency entropycoding (bit-plane based bit modeling and arithmetic encoding), andrealizes marked improvements over JPEG with regard to coding efficiency.

Wavelet transformation uses a technique wherein, basically, image datais taken as input which is subjected to horizontal direction filteringand vertical direction filtering, in which lowband components arehierarchically divided. At this time, reading and writing of data to andfrom memory, such as readout of image data, writing of frequencycoefficients generated as the result of filtering, readout of frequencycoefficients once more, and so forth, need to be performed at a highfrequency.

There has been recognized the need for a technique by which to performwavelet transformation at high speeds, since image signals have a greatamount of data. Also, a great number of techniques for externallywriting frequency coefficients to memory and reading these in again havebeen proposed (See Japanese Unexamined Patent Application PublicationNo. 10-283342).

SUMMARY OF THE INVENTION

However, techniques wherein frequency coefficients are written out toexternal memory and read in again have the problem than sufficientbandwidth cannot be obtained since data is exchanged between theexternal memory an wavelet transformation unit, so it has been difficultto perform wavelet transformation at high speeds.

Also, raising the clock (operating frequency) is effective in increasingthe speed of data between the external memory and wavelet transformationunit, but simple increasing of the clock not only results in the problemof increased power consumption; this is not readily handled by hardwaresuch as FPGA (Field Programmable Gate Array) and PLD (Programmable LogicDevice).

It has been recognized that there is a need to enable high-speed wavelettransformation without requiring external memory.

A wavelet transformation device according to an embodiment of thepresent invention, for performing wavelet transformation at a pluralityof levels as to image signals, comprises: a horizontal filtering unitfor subjecting the image signals to horizontal direction lowbandanalysis filtering and highband analysis filtering; and buffers whichare independent for each of the levels, for holding frequencycomponents, which are generated as the results of the horizontaldirection analysis filtering by the horizontal filtering unit, for eachof the levels.

The lowband components and highband components in the frequencycomponents obtained as the results of the horizontal direction analysisfiltering may be reordered and held in the buffer.

The wavelet transformation device may further comprise a reordering unitfor performing reordering of brightness signals and color differencesignals, which are elements of the image signals; with the horizontalfiltering unit subjecting the image signals reordered by the reorderingunit to horizontal direction lowband analysis filtering and highbandanalysis filtering.

The frequency components of the brightness signals and the frequencycomponents of the color difference signals generated as a result of thehorizontal analysis filtering performed by the horizontal filtering unitmay each be held separately in the buffers.

The wavelet transformation device may further comprise: a verticalfiltering unit for subjecting the frequency components, generated as aresult of the horizontal direction analysis filtering, that are held inthe buffers, to vertical direction lowband analysis filtering andhighband analysis filtering.

The vertical filtering unit may further comprise: a brightness signalvertical filtering unit for subjecting frequency components ofbrightness signals, which are elements of the image signals, to verticaldirection lowband analysis filtering and highband analysis filtering;and a color difference signal vertical filtering unit for subjectingfrequency components of color difference signals, which are elements ofthe image signals, to vertical direction lowband analysis filtering andhighband analysis filtering.

The brightness signal vertical filtering unit and the color differencesignal vertical filtering unit may be operated in parallel.

The wavelet transformation device may further comprise: a reorderingunit for performing reordering of the lowband components of thefrequency components of the brightness signals generated as a result ofthe vertical direction analysis filtering performed by the brightnesssignal vertical filtering unit, and the frequency components of thecolor difference signals generated as a result of the vertical directionanalysis filtering performed by the color difference signal verticalfiltering unit; with the horizontal filtering unit subjecting thelowband components reordered by the reordering unit to horizontaldirection lowband analysis filtering and highband analysis filtering.

Prior to reordering by the reordering unit, one of the brightness signalvertical filtering unit and the color difference signal verticalfiltering unit may stand by until the analysis filtering of the otherends.

The horizontal filtering unit may perform the horizontal directionlowband analysis filtering and highband analysis filtering to apredetermined number of levels.

The frequency components of the brightness signals and the frequencycomponents of the color difference signals generated as a result of thehorizontal direction analysis filtering performed by the horizontalfiltering unit may be separately held in the buffers.

The horizontal filtering unit and the vertical filtering unit mayperform analysis filtering on the lowest band frequency components in ahierarchical manner.

The horizontal filtering unit and the vertical filtering unit may berealized by a lifting scheme of the wavelet transformation.

The horizontal filtering unit may input the image signals in incrementsof lines, and perform the horizontal direction lowband analysisfiltering and highband analysis filtering each time the number ofsamples in the horizontal direction reaches a predetermined number; withthe vertical filtering unit performing the vertical direction lowbandanalysis filtering and highband analysis filtering each time the numberof lines in the vertical direction of the frequency component in theresults of the horizontal direction analysis filtering performed by thehorizontal filtering unit reach a predetermined number.

The image signals may be video signals comprising a plurality ofpictures, with the wavelet transformation device further comprising adetecting unit for detecting the end of each picture by detectingvertical synchronization signals of the video signals, and thehorizontal filtering unit and the vertical filtering unit performinganalysis filtering for each picture.

According to an embodiment of the present invention, a wavelettransformation method of a wavelet transformation device for performingwavelet transformation at a plurality of levels as to image signals,comprises the steps of: subjecting the image signals to horizontaldirection lowband analysis filtering and highband analysis filtering;and holding frequency components, which are generated as the results ofthe horizontal direction analysis filtering, for each of the levels, inbuffers which are independent for each of the levels.

With the above configuration, image signals are subjected to horizontaldirection lowband analysis filtering and highband analysis filtering.Frequency components, which are generated as the results of thehorizontal direction analysis filtering, are held for each of thelevels, in buffers which are independent for each of the levels. Thus,high speed wavelet transformation can be performed without requiringexternal memory.

According to an embodiment of the present invention, a wavelet inversetransformation device for performing wavelet inverse transformation asto frequency components, generated by a plurality of levels of wavelettransformations having been performed as to image signals, therebyreconstructing an image, comprises: horizontal filtering unit forsubjecting the frequency components to horizontal direction lowbandsynthesizing filtering and highband synthesizing filtering; and bufferswhich are independent for each of the levels except for the lowest band,for holding frequency components, which are generated as the results ofthe horizontal direction synthesizing filtering by the horizontalfiltering unit, for each of the levels.

The wavelet inverse transformation device may further comprise: avertical filtering unit for subjecting the frequency components tovertical direction lowband analysis filtering and highband analysisfiltering; with the horizontal filtering unit subjecting the frequencycomponents generated as a result of the vertical direction synthesizingfiltering to the horizontal direction lowband synthesizing filtering andhighband synthesizing filtering.

The vertical filtering unit and the horizontal filtering unit may berealized by a lifting scheme of the wavelet inverse transformation.

The horizontal filtering unit may input the frequency components inincrements of lines, and perform the horizontal direction lowbandsynthesizing filtering and highband synthesizing filtering each time thenumber of samples in the horizontal direction reaches a predeterminednumber; with the vertical filtering unit performing the verticaldirection lowband synthesizing filtering and highband synthesizingfiltering each time the number of lines in the vertical direction of thefrequency component in the results of the horizontal directionsynthesizing filtering performed by the horizontal filtering unit reacha predetermined number.

The image signals may be video signals comprising a plurality ofpictures, divided into a plurality of frequency components by performinganalysis filtering on the lowest band frequency components in ahierarchical manner; with the vertical filtering unit and the horizontalfiltering unit perform synthesizing filtering in a hierarchical mannerfrom, of a plurality of frequency components, a predetermined number offrequency components including the lowest band frequency components,ultimately generating a picture.

The wavelet inverse transformation device may further comprise avertical synchronizing signal insertion unit for inserting verticalsynchronizing signals between the pictures generated by the verticalfiltering unit and the horizontal filtering unit, thereby generatingvideo signals.

The vertical filtering unit, the horizontal filtering unit, and thebuffers, may be provided separately for brightness signals and for colordifference signals, which are elements of the image signals; with thevertical filtering unit, the horizontal filtering unit, and the buffers,for brightness signals, and the vertical filtering unit, the horizontalfiltering unit, and the buffers, for color difference signals, beingoperated in parallel.

According to an embodiment of the present invention, a wavelet inversetransformation method for performing wavelet inverse transformation asto frequency components generated by a plurality of levels of wavelettransformations being performed as to image signals, therebyreconstructing an image, comprises the steps of: subjecting thefrequency components to horizontal direction lowband synthesizingfiltering and highband synthesizing filtering; and holding frequencycomponents, which are generated as the results of the horizontaldirection synthesizing filtering by the horizontal filtering unit, foreach of the levels, in buffers which are independent for each of thelevels except for the lowest band.

With the above configuration, frequency components generated by aplurality of levels of wavelet transformations being performed as toimage signals are subjected to horizontal direction lowband synthesizingfiltering and highband synthesizing filtering. The frequency components,which are generated as the results of the horizontal directionsynthesizing filtering by the horizontal filtering unit, are held foreach of the levels, in buffers which are independent for each of thelevels except for the lowest band. Thus, high speed wavelet inversetransformation can be performed without requiring external memory.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a configuration example of anembodiment of a wavelet transformation device to which an embodiment ofthe present invention has been applied;

FIG. 2 is a flowchart for describing wavelet transformation processingwith the wavelet transformation device shown in FIG. 1;

FIG. 3 is a diagram for describing an example of horizontal analysisfiltering of image signals wherein Y and C have been interleaved;

FIG. 4 is a diagram for describing an example of performing verticalanalysis filtering separately for Y and C;

FIG. 5 is a diagram for describing an example of performing verticalanalysis filtering of Y and C, divided into lowband and highband aspreviously known, for the sake of comparison with FIG. 4;

FIG. 6 is a diagram illustrating the results of performing analysisfiltering to division level 1, for Y and C separately.

FIG. 7 is a diagram for describing an example of interleaving Y and Cfor division level 1 lowband components;

FIG. 8 is a diagram for describing an example of executing verticalanalysis filtering with Y and C interleaved, for the sake of comparisonwith FIG. 4;

FIG. 9 is a diagram illustrating an example of a timing chart in a casewherein vertical analysis filtering is performed separately for Y and C;

FIG. 10 is a diagram illustrating another example of a timing chart in acase wherein vertical analysis filtering is performed separately for Yand C;

FIG. 11 is a diagram for describing an example of a lifting scheme witha 5×3 analysis filter;

FIG. 12 is a diagram illustrating an example of a data array ofbrightness and color difference signals with the HDTV standard;

FIG. 13 is a diagram for describing analysis filtering in increments oflines;

FIG. 14 is a diagram for describing vertical filtering in division level1 analysis filtering;

FIG. 15 is a diagram illustrating the results of performing analysisfiltering to division level 2;

FIG. 16 is a diagram illustrating the results of performing analysisfiltering to division level 3 with an actual image;

FIG. 17 is a diagram for describing vertical synchronizing signals invideo signals;

FIG. 18 is diagram illustrating a configuration example relating to awavelet inverse transformation device corresponding to the wavelettransformation device shown in FIG. 1;

FIG. 19 is a flowchart for describing wavelet inverse transformationprocessing of the wavelet inverse transformation device shown in FIG.18;

FIG. 20 is a diagram for describing vertical synthesizing filtering;

FIG. 21 is a diagram for describing horizontal synthesizing filtering;

FIG. 22 is a diagram illustrating the results of performing synthesizingfiltering;

FIG. 23 is a diagram for describing another example of a lifting schemewith a 5×3 analysis filter;

FIG. 24 is a diagram for illustrating a configuration example of anembodiment of an image encoding device to which an embodiment of thepresent invention has been applied;

FIG. 25 is a flowchart for describing image encoding processing with theimage encoding device shown in FIG. 24;

FIG. 26 is a diagram illustrating a configuration example of anembodiment of an image decoding device corresponding to the imageencoding device shown in FIG. 24;

FIG. 27 is a flowchart for describing image decoding processing with theimage decoding device shown in FIG. 26;

FIG. 28 is a block diagram illustrating the configuration of an exampleof a digital triax system to which an embodiment of the presentinvention has been applied;

FIG. 29 is a block diagram illustrating the configuration of an exampleof a wireless transmission system to which an embodiment of the presentinvention has been applied;

FIG. 30 is a schematic drawing illustrating an example of applying thewireless transmission system shown in FIG. 29 to a home gaming console;and

FIG. 31 is a block diagram illustrating a configuration example of anembodiment a computer to which an embodiment of the present inventionhas been applied.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is a diagram illustrating a configuration example of anembodiment of a wavelet transformation device to which an embodiment ofthe present invention has been applied. The wavelet transformationdevice 1 is a band analysis device which takes image data as input, andperforms horizontal direction filtering and vertical directionfiltering, in which lowband components are hierarchically divided to apredetermined division level (in the example shown in FIG. 1, todivision level 4).

The wavelet transformation device 1 shown in FIG. 1 is configured of aninterleaving unit 10, a horizontal analysis filter unit 11, level 1buffer 12, level 2 buffer 13, level 3 buffer 14, level 4 buffer 15,selector 16, Y (brightness) vertical analysis filter unit 17, C (colordifference) vertical analysis filter unit 18, interleaving unit 19,horizontal analysis filter unit 20, and control unit 21.

Brightness signals (brightness components signals) D10 and colordifferent signals (color difference component signals) D11, which areelements of image signals, are input to the interleaving unit 10. Notethat hereinafter, brightness (components) may be referred to as “Y” asappropriate, and color difference (components) as “C”. The interleavingunit 10 interleaves the brightness signals D10 and color differencesignals D11. In the event of image signals wherein Y and C have alreadybeen interleaved are input, as with the video signals described laterwith reference to FIG. 12, there is no need to the processing by theinterleaving unit 10, as a matter of course. The interleaved imagesignals D12 are input to the horizontal analysis filter unit 11.

The horizontal analysis filter unit 11 subjects the image signals D12,wherein Y and C have been interleaved, to lowband analysis filtering andhighband analysis filtering in the horizontal direction of the divisionlevel 1, and generates frequency components coefficients made up oflowband component coefficients and highband component coefficientsobtained as the result of the horizontal analysis filtering (hereafteralso referred to as lowband component, highband component and frequencycomponents, as appropriate).

Now, the horizontal analysis filter unit 11 performs lowband analysisfiltering and highband analysis filtering in the horizontal direction onthe Y or C data, while alternately reading out the Y and C data situatedon the baseband in stepping stone fashion, from an unshown internalmemory (or register).

The level 1 buffer 12 stores and holds the results of the division level1 horizontal analysis filtering. That is to say, the level 1 buffer 12separately stores and holds the Y frequency components (lowbandcomponents and highband components) and the C frequency components(lowband components and highband components), obtained as results of thedivision level 1 horizontal analysis filtering by the horizontalanalysis filter unit 11. Once data (frequency components) for apredetermined number of vertical lines enabling vertical directionanalysis filtering has been accumulated in the level 1 buffer 12, thenumber of vertical lines worth of frequency components D14 are read outvia the selector 16.

The level 2 buffer 13 stores and holds the results of the division level2 horizontal analysis filtering. That is to say, the level 2 buffer 13separately stores and holds the Y frequency components (lowbandcomponents and highband components) and the C frequency components(lowband components and highband components), obtained as results of thedivision level 2 horizontal analysis filtering by the horizontalanalysis filter unit 20. Once data for a predetermined number ofvertical lines enabling vertical direction analysis filtering has beenaccumulated in the level 2 buffer 13, the number of vertical lines worthof frequency components D15 are read out via the selector 16.

The level 3 buffer 14 stores and holds the results of the division level3 horizontal analysis filtering. That is to say, the level 3 buffer 14separately stores and holds the Y frequency components (lowbandcomponents and highband components) and the C frequency components(lowband components and highband components), obtained as results of thedivision level 3 horizontal analysis filtering by the horizontalanalysis filter unit 20. Once data for a predetermined number ofvertical lines enabling vertical direction analysis filtering has beenaccumulated in the level 3 buffer 14, the number of vertical lines worthof frequency components D16 are read out via the selector 16.

The level 4 buffer 15 stores and holds the results of the division level4 horizontal analysis filtering. That is to say, the level 4 buffer 15separately stores and holds the Y frequency components (lowbandcomponents and highband components) and the C frequency components(lowband components and highband components), obtained as results of thedivision level 4 horizontal analysis filtering by the horizontalanalysis filter unit 20. Once data (frequency components) for apredetermined number of vertical lines enabling vertical directionanalysis filtering has been accumulated in the level 4 buffer 15, thenumber of vertical lines worth of frequency components D17 are read outvia the selector 16.

Under control of the Y vertical analysis filter unit 17 and C verticalanalysis filter unit 18, the selector 16 selects from the level 1 buffer12 through level 4 buffer 15, the output of the corresponding divisionlevel buffer, and outputs the selected output to the Y vertical analysisfilter unit 17 and C vertical analysis filter unit 18 as frequencycomponents D18.

The Y vertical analysis filter unit 17 and C vertical analysis filterunit 18 perform Y and C vertical analysis filtering on the predeterminednumber of vertical lines worth of frequency components D18 from theselector 16.

That is to say, the Y vertical analysis filter unit 17 reads out thepredetermined number of vertical lines worth of frequency components D18for Y from the buffer of the corresponding level, performs Y verticaldirection lowband analysis filtering and highband analysis filtering,and of the frequency components obtained as a result of the Y verticalanalysis filtering, outputs only the lowband component D19 which islowband in both the horizontal direction and vertical direction for Y,to the interleaving unit 19, and outputs other highband components D23of Y externally from the wavelet transformation device 1 (hereafterreferred to simply as “external”).

Also, the C vertical analysis filter unit 18 reads out the predeterminednumber of vertical lines worth of frequency components D18 for C fromthe buffer of the corresponding division level, performs C verticaldirection lowband analysis filtering and highband analysis filtering,and of the frequency components obtained as a result of the C verticalanalysis filtering, outputs only the lowband component D20 which islowband in both the horizontal direction and vertical direction for C,to the interleaving unit 19, and outputs other highband components D24of C externally.

The interleaving unit 19 interleaves the Y lowband components D19 fromthe Y vertical analysis filter unit 17 and the C lowband components D20from the C vertical analysis filter unit 18. The interleaved lowbandcomponents D21 are input to the horizontal analysis filter unit 20.

The horizontal analysis filter unit 20 has basically the sameconfiguration as that of the horizontal analysis filter unit 11, exceptthat the division level of the frequency components to be processeddiffers. That is to say, with the horizontal analysis filter unit 20, Yand C lowband components D21 existing on the baseband in steppingstonefashion are alternately read out by the horizontal analysis filter unit20 from unshown built-in memory, and horizontal direction lowbandanalysis filtering and highband analysis filtering are performedalternately on the Y and C lowband components D21.

The horizontal analysis filter unit 20 then stores and holds frequencycomponents (lowband components and highband components) D22 which areresults of the horizontal analysis filtering, in the corresponding levelbuffer (one of the level 2 buffer 13 through level 4 buffer 15).

The control unit 21 is configured of a microcomputer or the likeincluding, for example, a CPU (Central Processing Unit), ROM (Read OnlyMemory), and RAM (Random Access Memory), and controls the processing ofthe units of the wavelet transformation device 1 by executing varioustypes of programs.

Next, the operations of the wavelet transformation device 1 shown inFIG. 1 will be described with reference to the flowchart in FIG. 2. Thatis to say, FIG. 2 illustrates the wavelet transformation processingexecuted by the wavelet transformation device 1.

Image signals are input to the interleaving unit 10 externally (e.g.,from a later-described video camera unit 303 shown in FIG. 28). In stepS11, the interleaving unit 10 determines whether or not to performinterleaving. For example, in the event that brightness signals D10 andcolor difference signals D11 which are elements of the image signals areinput, in step S11 determination is made to perform interleaving, theflow proceeds to step S12, and the interleaving unit 10 interleaves thebrightness signals D10 and color difference signals D11 in internalmemory. The interleaved image signals D12 are input to the horizontalanalysis filter unit 11.

Also, in the event that image signals wherein Y and C are interleaved,as with the video signals described later with reference to FIG. 12, areinput (i.e., signals equivalent to image signals D12), in step S11determination is made to not perform interleaving, and the image signalsD12 wherein Y and C are interleaved are input to the horizontal analysisfilter unit 11 via the interleaving unit 10 without change. The flowthen skips step S12, and proceeds to step S13. In step S13, thehorizontal analysis filter unit 11 performs division level 1 horizontalanalysis filtering on the image signals D12 wherein Y and C areinterleaved.

That is to say, the horizontal analysis filter unit 11 has unshowninternal memory (or a register), and the input image signals wherein Yand C have been interleaved are rendered in the internal memory as shownin FIG. 3. Note that with the example in FIG. 3, a baseband image ofimage signals wherein Y and C, represented by squares, are alternatelyrendered in memory, is illustrated.

The horizontal analysis filter unit 11 reads out Y data of apredetermined number (3 samples in the case of FIG. 3) or C data of apredetermined number situated in steppingstone fashion on the basebandrendered in the internal memory, while shifting the position, andalternately performs Y horizontal direction lowband analysis filteringand highband analysis filtering, and C horizontal direction lowbandanalysis filtering and highband analysis filtering.

The horizontal direction data is readily rendered to memory addresses.Accordingly, horizontal analysis filtering is readily executed in orderat the horizontal analysis filter unit 11 while alternately reading outY data and C data from the memory.

In step S14, the horizontal analysis filter unit 11 stores the frequencycomponents obtained as a result of the division level 1 horizontalanalysis filtering to the corresponding level buffer (in this case, thelevel 1 buffer 12).

At this time, the horizontal analysis filter unit 11 interleaves the Yhighband components (H) and lowband components (L) which are the resultsof the Y horizontal analysis filtering, and stores in the level 1 buffer12 separately from C, and then interleaves the C highband components andlowband components which are the results of the C horizontal analysisfiltering, and stores in the level 1 buffer 12 separately from Y, asshown in FIG. 4.

With known arrangements, the results of the horizontal analysisfiltering were divided into the highband components and lowbandcomponents, and thus stored in the buffer, for both Y and for C, asshown in FIG. 5. However, mapping the highband components and lowbandcomponents to different addresses in the buffer as with the knownarrangements requires a separate controller for distribution thereof.

Conversely, with the example shown in FIG. 4 according to the presentembodiment, the Y highband components (H) and lowband components (L) arealternately stored in the level 1 buffer 12, and the C highbandcomponents (H) and lowband components (L) are alternately stored ataddresses different from Y. That is to say, the highband components andlowband components which are the results of the horizontal analysisfiltering are stored in the level 1 buffer 12 in an interleaved mannerby the horizontal analysis filter unit 11 for Y and C each, so at thetime of reading out the frequency components stored in the level 1buffer 12, all that is to necessary is to read out from the front of thelevel 1 buffer 12, thereby simplifying control.

Now, returning to FIG. 2, upon the frequency components obtained as aresult of the horizontal analysis filtering being accumulated in thelevel 1 buffer 12 shown in FIG. 4 for a predetermined number of verticallines (three lines in the case shown in FIG. 4) whereby verticalanalysis filtering can be performed, for example, in step S15 the Yvertical analysis filter unit 17 and C vertical analysis filter unit 18read out the frequency components of the necessary number of verticallines from the level 1 buffer 12, by controlling the selector 16 so asto select the output of the level 1 buffer 12. The frequency componentsread out are input to the Y vertical analysis filter unit 17 and Cvertical analysis filter unit 18 as Y and C frequency components D18,via the selector 16.

In step S16, the Y vertical analysis filter unit 17 performs Y verticalanalysis filtering of the corresponding division level (in the currentcase, the division level 1) on a predetermined number of vertical linesworth of Y frequency components (three lines worth in the case of FIG.4). In step S17, the C vertical analysis filter unit 18 performs Cvertical analysis filtering of the corresponding division level (in thecurrent case, the division level 1) on a predetermined number ofvertical lines worth of C frequency components (three lines worth in thecase of FIG. 4). The processing of steps S16 and S17 are executed inparallel; these will be described later in detail with reference to FIG.8.

As a result of the division level 1 Y and C vertical analysis filtering(i.e., the results of division level 1 Y and C analysis filtering),frequency components of four frequency components are generated for Yand C, made up of the lowband component (1LL) coefficient and highbandcomponent (1HH, 1LH, 1HL) coefficients, as shown in FIG. 6. Note that inthe example shown in FIG. 6, the order of “L” and “H” indicate thebandwidth (lowband or highband) as the result of performing horizontalanalysis filtering previously, and the bandwidth (lowband or highband)as the result of performing vertical analysis filtering, in that order.Further, the numeral in front of the “L” or “H” indicates the divisionlevel.

This is the division level 1 analysis filtering, whereby Y lowbandcomponents (1LL) D19 and highband components (1HH, 1LH, 1HL) D23 aregenerated at the Y vertical analysis filter unit 17, and C lowbandcomponents (1LL) D20 and highband components (1HH, 1LH, 1HL) D24 aregenerated at the C vertical analysis filter unit 18, as a result. Ofthese, only the lowband components (1LL) are analyzed again to the setdivision level (final level), but the highband components are notanalyzed any further. That is to say, the lowband components are furtherdivided to the final level, so the final level can also be said to bethe lowest band level outputting the lowest band.

In step S18, the Y vertical analysis filter unit 17 and C verticalanalysis filter unit 18 determine whether or not the analysis filteringhas been computed to the final level of the set division levels (in thecase shown in FIG. 1, division level 4). In the case of division level1, the flow has not reached the final level yet, so the processingproceeds to step S19.

In step S19, the Y vertical analysis filter unit 17 and C verticalanalysis filter unit 18 output the Y highband components (1HH, 1LH, 1HL)D23 and C highband components (1HH, 1LH, 1HL) D24 externally (e.g., tothe quantizing unit 112 described later, shown in FIG. 24).

On the other hand, the Y lowband components (1LL) D19 and C lowbandcomponents (1LL) D20 are output to the interleaving unit 19.Accordingly, in step S20, the interleaving unit 19 interleaves the Ylowband components (1LL) D19 and C lowband components (1LL) D20, andinputs the interleaved lowband components D21 to the horizontal analysisfilter unit 20.

that is to say, as shown in FIG. 7, the lowband components (1LL) D19 ofthe Y frequency components and the lowband components (1LL) D20 of the Cfrequency components are alternately interleaved and synthesized. Notethat in the example shown in FIG. 7, the size of the lowband components(1LL) following interleaving of Y and C shown toward the bottom isillustrated as being the same size as the lowband components (1LL) of Yor C alone before interleaving, indicated by the dotted line, but inreality, the size is twice that of the lowband components of Y or Calone.

In step S21, the horizontal analysis filter unit 20 performs horizontalanalysis filtering (horizontal direction lowband analysis filtering andhighband analysis filtering) of the corresponding division level (inthis case, division level 2) on the lowband components D21 wherein Y andC have been interleaved, and generates lowband components and highbandcomponents which are the results of the horizontal analysis filtering.Note that the processing in step S21 is basically the same as theprocessing in the above-described step S13, with the only differencebeing the division level of the frequency components to be processed.

Following step S21, the flow returns to step S14, and the subsequentprocessing is repeated. That is to say, in step S14, the horizontalanalysis filter unit 20 stores and holds the frequency components(lowband and highband components) D22 obtained as the result ofhorizontal analysis filtering, in the buffer of the corresponding level(in this case, level 2 buffer 13).

Upon the frequency components obtained as a result of the horizontalanalysis filtering being accumulated in the level 2 buffer 13 for apredetermined number of vertical lines whereby vertical analysisfiltering can be performed, in step S15 the frequency components of thenecessary number of vertical lines are read out from the level 2 buffer13, and are input to the Y vertical analysis filter unit 17 and Cvertical analysis filter unit 18, via the selector 16. In step S16,division level 2 Y vertical analysis filtering is performed as to the Yfrequency components of the predetermined number of lines, and in stepS17, division level 2 C vertical analysis filtering is performed as tothe C frequency components of the predetermined number of lines.

As a result of the division level 2 Y and C vertical analysis filtering,four frequency components are generated, made up of the lowbandcomponent (2LL) coefficient and highband component (2HH, 2LH, 2HL)coefficients, for Y and C each. In step S18, determination is made thatthe flow has not reached the final level yet, so in step S19 thehighband components (2HH, 2LH, 2HL) D23 are externally output. In stepS20, the Y and C of the lowband components (2LL) are interleaved, and instep S21, division level 2 horizontal analysis filtering is performed onthe lowband components (2LL) wherein the Y and C have been interleaved,thereby generating lowband components and highband components which arethe results of the horizontal analysis filtering, the flow returns tostep S14 again, the generated lowband components and highband componentsare stored and saved in the level 3 buffer 14, and the subsequentprocessing is repeated until the final level of the preset divisionlevels.

The above-described series of processing is performed in the same mannerup to the Y and C vertical analysis filtering at the final level(division level 4) of the preset division levels. Subsequently, in stepS18, determination is made that the final level has finished, and theflow proceeds to step S22.

In step S22, the Y vertical analysis filter unit 17 and C verticalanalysis filter unit 18 externally output the final level brightnessfrequency components (4LL, 4HL, 4LH, 4HH) D23 and the final level colordifference frequency components (4LL, 4HL, 4LH, 4HH) D24. Thus, thedivision level 4 image signal wavelet transformation ends.

As described above, the wavelet transformation device 1 shown in FIG. 1has buffers for each division level from level 1 to a predeterminednumber of levels, and stores the horizontal analysis filtering resultsin buffers of each division level while performing the horizontalanalysis filtering. Accordingly, vertical direction filtering can beperformed while reading out the results of the horizontal analysisfiltering from the buffer of each division level. That is to say,horizontal direction and vertical direction analysis filtering can beperformed simultaneously in parallel.

Thus, wavelet transformation can be performed at high speed for movingimages and images with high resolution, as well.

Also, internal memory handles the buffers for each division level fromlevel 1 to a predetermined level, so there is no need to configureexternal memory as with the arrangement described in Japanese UnexaminedPatent Application Publication No. 10-283342.

Accordingly, there is no need to exchange data with external memory, andwavelet transformation can be performed at higher speeds than whenaccessing external memory. Consequently, there is no need to raise theclock frequency in order to increase speed of data between the externalmemory and the wavelet transformation device, thereby conservingelectric power.

Also, horizontal direction analysis filtering is performed as tofrequency components wherein Y and C are interleaved, so the horizontalanalysis filter unit can be a single configuration, markedlycontributing to reduction in the size of the hardware. This datainterleaved in the horizontal direction can be readily rendered toregisters or memory, and further, can be read and written at high speed,thereby contributing to higher speed of wavelet transformation.

Further, as described above with reference to FIGS. 4 and 5, thehighband components and lowband components which are the results of thehorizontal analysis filtering are interleaved for each of Y and C, andstored in buffers of corresponding levels, Y and C being storedseparately, so at the time of reading out, all that is necessary is toread out from the front of the buffer of that level, thereby simplifyingcontrol.

Also, vertical direction analysis filtering is performed separately forY and C, so there is no need for the massive memory capacity which isnecessary in the event of not performing the vertical direction analysisfiltering shown in FIG. 8 separately for Y and C, and drastic increasesin cost can be prevented. Further, the need for extra processing timecan be prevented, as well.

Now, FIG. 8 will be used to describe a case wherein the verticaldirection analysis filtering is not performed separately for Y and C,i.e., wherein Y and C are interleaved as with the horizontal analysisfilter units 11 and 19, and then Y and C vertical analysis filtering isalternately performed while shifting, in order to compare such anarrangement with that of an embodiment of the present invention.

In the example in FIG. 8 is shown an example of a line buffer whereinterleaved Y and C are rendered. In the event that the image isconfigured of n lines for example, first, the brightness component Y1 ofthe first line is rendered at the line buffer, next, the colordifference component C1 of the first line is rendered, the brightnesscomponent Y2 of the second line is rendered, next, the color differencecomponent C2 of the second line is rendered, the brightness component Y3of the third line is rendered, next, the color difference component C3of the third line is rendered, and so on, until the brightness componentYn of the n'th line is rendered, and next, the color differencecomponent Cn of the n'th line is rendered, and so the interleaved Y andC are rendered to the line buffer.

That is to say, in the case of processing Y and C separately as with theY vertical analysis filter unit 17 and C vertical analysis filter unit18 shown in FIG. 1, all that is necessary is to read out Y and Cseparately, but in order to process Y and C together, there is the needto interleave and render Y and C, requiring an extra line buffer for C(or for Y).

Also, the vertical analysis filtering is performed at the point that thenumber of lines necessary for vertical analysis filtering to beperformed (e.g., 3 lines) is accumulated, so in the example in FIG. 8,in actual operations, Y vertical analysis filtering is performed at thepoint that the brightness component Y3 the fifth line from the top isaccumulated, C vertical analysis filtering is performed at the pointthat the color difference component C3 the sixth line from the top isaccumulated, and so on, with Y and C vertical analysis filtering beingalternately performed.

Conversely, with the case of the Y vertical analysis filter unit 17 andC vertical analysis filter unit 18 shown in FIG. 1, Y and C verticalanalysis filtering are performed in parallel, so the processing speedper time unit is faster than that of the example in FIG. 8.

AS described above, an arrangement wherein Y and C vertical analysisfiltering is performed after interleaving Y and C is advantageous inthat only one vertical analysis filter unit is needed, but this requiresan extra line buffer for C (or for Y) as compared with an embodiment ofthe invention. In the even of configuring this line buffer with built-inmemory, an extremely great amount of memory is necessary of images withhigh horizontal resolution (e.g., 1920 pixels in the case of HDTV (HighDefinition TeleVision), markedly increasing costs. Also, in the exampleshown in FIG. 8, Y and C must be alternately processed with a singlevertical analysis filter unit, so more processing time is required ascompared to the case of performing processing with two vertical analysisfilter units, one for Y and one for C as with an embodiment of theinvention.

Thus, an arrangement wherein Y and C vertical analysis filtering isperformed after interleaving Y and C is more disadvantageous thanadvantageous. Conversely, the case of performing processing with twovertical analysis filter units, one for Y and one for C, as with anembodiment of the invention, an extra line buffer is not necessary, somarked increases in cost can be suppressed, and further, the processingcan be performed in parallel with the two vertical analysis filterunits, so processing time can be sped up.

As described above, Y vertical analysis filtering with the Y verticalanalysis filter unit 17, and C vertical analysis filtering with the Cvertical analysis filter unit 18, are performed in parallel, butgenerally, the brightness components have a greater data amount thancolor difference components, so the filtering execution time tends to belonger, and accordingly, even if these two are started at the same time,they will not end at the same time.

Accordingly, of the vertical analysis filtering results of Y and C, thehighband components D23 and D24 are not subjected to horizontal analysisfiltering again, so as shown in FIG. 9, one keeps waiting for the other.

That is to say, in the example shown in FIG. 9, after the processing ofthe first line of brightness components Y1 started at the same time asthe processing of first line of color difference components C1,processing of the second line of brightness components Y2, processing ofthe third line of brightness components Y3, and so on through processingof the n'th line of brightness components Yn, is sequentially performed,thereby completing the Y vertical analysis filtering.

Also, processing of the second line of color difference components C2starts after the processing of the first line of color differencecomponents C1 without waiting for the processing of the first line ofbrightness components Y1, which started at the same time, to end, andthen after the processing of the second line of color differencecomponents C2, processing of the third line of color differencecomponents C3, and so on, is performed to the processing of the n'thline of color difference components Cn, so consequently, the C verticalanalysis filtering is ended before the Y vertical analysis filteringends.

On the other hand, of the Y and C vertical analysis filtering results,the lowband components D19 and D20 then need to be interleaved at theinterleaving unit 19, so there is the need to match the output timing ofboth of the data, as shown in FIG. 10.

That is to say, in the example shown in FIG. 10, as with the exampleshown in FIG. 9, after the processing of the first line of brightnesscomponents Y1 started at the same time as the processing of first lineof color difference components C1, processing of the second line ofbrightness components Y2, processing of the third line of brightnesscomponents Y3, and so on through processing of the n'th line ofbrightness components Yn, is sequentially performed, thereby completingthe Y vertical analysis filtering.

Conversely, processing of the second line of color difference componentsC2 is not performed immediately after processing of the first line ofcolor difference components C1, but after awaiting the processing of thefirst line of brightness components Y1 started at the same time as theprocessing of the first line of color difference components C1 to end.Processing of the third line of color difference components C3 is notperformed immediately after processing of the second line of colordifference components C2, but after awaiting the processing of thesecond line of brightness components Y2 started at the same time as theprocessing of the second line of color difference components C2 to end.Finally, processing of the n′th line of color difference components Cnis not performed immediately after processing of the n−1′th line ofcolor difference components C n−1, but after awaiting the processing ofthe n−1′th line of brightness components Y n−1 started at the same timeas the processing of the n−1′th line of color difference components Cn−1 to end.

As described above, in order to perform the vertical direction analysisfiltering of Y and C separately (in parallel), there is the need tomatch the output time of the lowband components obtained as results of Yand C. Thus, interleaving of the Y and C lowband components describedabove with reference to FIG. 7 can be made to proceed smoothly.

Next, the computation method in the above-described analysis filteringwill be described in detail. The most common computation method inanalysis filtering computation methods is a method called convolutioncomputation. This convolution computation the most basic way to realizedigital filters, with convolution multiplication being performed onactual input data on filter tap coefficients. However, with convolutioncomputation, if the tap length is great, there cases wherein thecalculation load increases accordingly.

Wavelet transformation lifting, introduced in the paper “W. Swelden,‘The lifting scheme: A custom-design construction of biorthogonalwavelets’, Appl. Comput. Harmon. Anal., Vol 3, No. 2, pp. 186-200,1996”, is a known technique for handling this.

FIG. 11 illustrates a lifting scheme of a 5×3 analysis filter used withthe JPEG (Joint Photographic Experts Group) 2000 standard as well.Analysis lifting in a case of applying the lifting technique to this 5×3analysis filter will be described.

In the example shown in FIG. 11, the top tier, middle tier, and bottomtier respectively represent a pixel row of an input image, highbandcomponent output, and lowband component output. The top tier is notrestricted to a pixel row of an input image, and may be coefficients(frequency components) obtained from previous analysis filtering. Notethat here, top tier is a pixel row of an input image, with the solidsquares representing even pixels or lines, and the solid circlesrepresenting odd pixels or lines.

As the first stage, highband component coefficients are generated fromthe input pixel row, as shown in the following Expression (1).coefficient d _(i) ¹ =d _(i) ⁰−1/2(s _(i) ⁰ +s _(i+1) ⁰)  (1)

Next, as the second stage, the lowband component coefficients aregenerated using the generated highband component coefficients andodd-numbered pixels of the input pixel row, as shown in the followingExpression (2).coefficient s _(i) ¹ =s _(i) ⁰+1/4(d _(i−1) ¹ +d _(i) ¹)  (2)

Thus, with the analysis filtering, first, highband componentcoefficients are generated, following which lowband componentcoefficients are generated. The tow types of filter banks used at thistime can be realized with addition and shift computations alone, as canbe seen from Expression (1) and Expression (2). Also, with a Ztransformation expression, the filter has no more than two taps, asshown in the following Expression (3). In other words, two taps cansuffice where five were necessary, and accordingly, the amount ofcalculations can be markedly reduced. Thus, this lifting technique willbe applied to the horizontal analysis filtering and the verticalanalysis filtering with the wavelet transformation device 1 as well.P(z)=(1+z ⁻¹)/2, U(z)=(1+z ⁻¹)/4  (3)

Now, while the above description has been made regarding theconfiguration and operations of the wavelet transformation device 1shown in FIG. 1 using common image signals; next, a case will bedescribed wherein video signals, which are moving images, are input tothe wavelet transformation device 1.

Video signals are normally stipulated by standards, and televisionbroadcast signals which are generally used in Japan, the USA, andseveral other countries, are NTSC (National Television StandardsCommittee) signals. Also, HDTV signals are standardized under a standardknown as SMPTE274M, by a USA standardization entity, SMPTE (The Societyof Motion Picture and Television Engineers). Description will be madehere regarding HDTV signals (resolution of 1920×1080).

FIG. 12 illustrates the components of HDTV video signal data. Of thevideo signals, the number of real sample data of brightness component Yis 1920 samples per line, with sample data of EAV (End of Active Video),SAV (Start of Active Video) being positioned before the Y real sampledata. These are made up of a total of 280 samples. This configuration isalso the same for color difference components Cb and Cr, but the formatis 4:2:2 with the number of real sample data of Cb and Cr each beinghalf of Y, so the total of Cb and Cr is the same as Y.

Multiplexing the Y and CB, Cr generates data of a total of 560 samplesfor EAV and SAV, and a total of 3840 samples for Y, Cb, and Cr. Thus,HDTV SMPTE274M standard (normally called “HD-SDI (High Definition SerialData Interface) standard”) video signals already have Y and Cinterleaved. Accordingly, this multiplexed sample data is equivalent tothe image signals D12 shown in the example in FIG. 1. That is opt say,the image signals D12 are input to the interleaving unit 10 shown inFIG. 1, and as described above with step S11 in FIG. 2, the imagesignals D12 are input to the horizontal analysis filter unit 11 with nochange thereto. Description will be made below assuming this situation.

In the event that video signals are input as image signals to thewavelet transformation device 1, the video signals are input as 60fields being input per second, or each picture being input at 1/60seconds, so the wavelet transformation processing described earlier withreference to FIG. 2 must be completed in this short time. That is tosay, wavelet transformation must be completed at high speed.

One way of dealing with this is to input video signals (image signalsD12) at the horizontal analysis filter unit 11, and at the point thatthe number of columns in the horizontal direction (number of samples)reach the predetermined number, to immediately perform horizontaldirection lowband analysis filtering and highband analysis filtering.Note that while the example shown in FIG. 13 is a case regardingbrightness signals Y, color difference signals C are also processed inthe same way. Further, while description will be omitted, this holdstrue for the horizontal analysis filter unit 20 as well.

For example, the horizontal analysis filter unit 11 stands by until 2 Mcolumns of image signals D12 wherein Y and C have been interleaved isinput and rendered at the internal memory. The value of M corresponds tothe number of taps for horizontal analysis filtering, and the greaterthe number of taps is, the greater the value of M is, accordingly. Notethat the image signals D12 have Y and C interleaved, so storing of twicethe number of columns of M is awaited.

The horizontal analysis filter unit 11 immediately performs horizontalanalysis filtering at the point that M columns worth of Y signals areaccumulated in the built-in memory. That is to say, the horizontalanalysis filter unit 11 sequentially reads out the M columns worth(e.g., M=3 in the case of FIG. 3) of Y from the built-in memory, andperforms Y horizontal direction lowband analysis filtering and highbandanalysis filtering. The Y lowband components and highband componentswhich are the results of the horizontal analysis filtering areinterleaved, and stored in the level 1 buffer 12. The Y lowbandcomponents and highband components which are stored in the level 1buffer 12 are read out and input to the Y vertical analysis filter unit17 at the point that the number of lines reaches N lines.

The Y vertical analysis filter unit 17 immediately performs verticaldirection lowband analysis filtering and highband analysis filtering atthe point that N lines worth (e.g., N=3 in the case of FIG. 4) of Ylowband and highband components are accumulated. The value of Ncorresponds to the number of taps for vertical analysis filtering, andthe greater the number of taps is, the greater the value of N is,accordingly. As shown in FIGS. 13 and 14, this Y vertical analysisfiltering generates, as the results of the vertical analysis filtering,lowband components (1LL) D19, and highband components (1HL, 1LH, 1HH)D23.

Now, while description will be omitted, following the Y horizontalanalysis filtering, the horizontal analysis filter unit 11 performs Chorizontal analysis filtering while shifting position in the same way asthe Y horizontal analysis filtering, and the C vertical analysis filterunit 18 performs C vertical analysis filtering on the results ofhorizontal analysis filtering in the same way as with the Y verticalanalysis filtering, in parallel to the processing of the Y verticalanalysis filter unit 17, thereby generating the lowband components (1LL)D20 and highband components (1HL, 1LH, 1HH) D24 which are the C verticalanalysis filtering results.

Following the vertical analysis filtering, The Y lowband components D19and the C lowband components D20 are interleaved at the interleavingunit 19, and at the point that as many columns as necessary to enablehorizontal direction analysis filtering of the lowband components D21wherein Y and C have been interleaved are accumulated in the memory ofthe interleaving unit 19, the horizontal analysis filter unit 20immediately performs division level 2 horizontal analysis filtering. Thereason that the lowband components are repeatedly analyzed in this wayis that the greater portion of energy of image signals are concentratedin the lowband components.

The horizontal analysis filter unit 20 sequentially reads out M columnsfrom the internal memory, and performs Y horizontal direction lowbandanalysis filtering and highband analysis filtering, as the divisionlevel 2 horizontal analysis filtering. The Y lowband components andhighband components which are the results of the horizontal analysisfiltering are interleaved, and stored in the level 2 buffer 13. Whiledescription will be omitted here, this is also true for C.

The vertical analysis filter unit 18 immediately performs Y verticaldirection lowband analysis filtering and highband analysis filtering atthe point that N/2 lines worth of lowband and highband components areaccumulated at the level 2 buffer 13, as shown in FIG. 14. As shown inFIG. 15, this vertical filtering generates lowband components (2LL) andhighband components (2HL, 2LH, 2HH). That is to say, in the exampleshown in FIG. 15, the frequency components of the division level 1 1LLare divided into the four frequency components of 2LL, 2HL, 2LH, AND2HH.

As described above, wavelet transformation which is repeatedly performedunit a predetermined division level is repeatedly performed whileinputting subsequently video signals to the end of one picture of videosignals, whereby one image is subjected to band division to apredetermined division level.

In the event of further increasing the number of division levels,analysis filtering can be repeatedly performed on the lowbandcomponents. FIG. 16 is a diagram illustrating an example wherein anactual image has been divided to division level 3 by analysis filteringwherein N=4.

That is to say, with this image, in the division level 1 verticalanalysis filtering, as soon as four lines worth of frequency componentsare accumulated, vertical analysis filtering is performed; in thedivision level 2 vertical analysis filtering, as soon as two lines worthof frequency components are accumulated, vertical analysis filtering isperformed; and in the division level 3 vertical analysis filtering, assoon as one line worth of frequency components are accumulated, verticalanalysis filtering is performed; whereby it can be understood thatdivision has been performed to division level 3.

As described above, analysis filtering is performed as soon as apredetermined number of columns or a predetermined number of lines worthof frequency components are accumulated, so analysis filtering of videopictures of one picture worth can be effectively performed. That is tosay, wavelet transformation can be performed at high speed.

Also, performing analysis filtering for obtaining coefficient data forat least one line of lowband components, multiple times in stages forall lines of the entire screen, enables decoded image to be obtainedwith little delay in a system wherein, for example,post-wavelet-transformation frequency components are encoded andtransferred, and decoded, as described later with reference to FIG. 28.

Also, analysis filtering performed on video signals as described aboveis performed in increments of pictures (fields or frames) making upvideo signals, so there is the need to detect the end of a picture, andstop and reset the analysis filtering operations. In this case, whileillustrating in the drawings will be omitted, the wavelet transformationdevice 1 is configured having a vertical synchronizing signal detectingunit built in for detecting vertical synchronizing signals in the videosignals, provided to the interleaving unit 10, for example.

FIG. 17 is a signal distribution diagram of SMPTE274M standardinterlaced signals. The upper half shows the first field, and the lowerhalf shows the second field. There are 22 lines worth of verticalsynchronizing signals are at the front of the first field, and 23 linesworth of vertical synchronizing signals are at the front of the secondfield.

Accordingly, the wavelet transformation device 1 has a verticalsynchronizing signal detecting unit for detecting vertical synchronizingsignals in the video signals, built in a the interleaving unit 10 forexample, and detects the vertical synchronizing signals with thebuilt-in vertical synchronizing signal detecting unit.

Thus, the end of a picture can be readily detected, and analysisfiltering operations can be stopped upon detection. That is to say,analysis filtering can be performed on video signals in increments ofpictures (fields or frames) making up the video signals.

FIG. 18 is a diagram illustrating a configuration example of anembodiment of a wavelet inverse transformation device corresponding tothe wavelet transformation device shown in FIG. 1. The wavelet inversetransformation device 51 is a band synthesizing device which takesfrequency components obtained by image signals being subjected towavelet transformation and divided to a predetermined level (in the caseshown in FIG. 18, division level 4) by the wavelet transformation device1 shown in FIG. 1. Of course, if the number of wavelet transformationdivision levels differs, wavelet inverse transformation is performedcorresponding to the number of division levels.

The wavelet inverse transformation device 51 shown in FIG. 18 isconfigured of a level 3 buffer 61, level 2 buffer 62, level 1 buffer 63,selector 64, vertical synthesizing filter unit 65, horizontalsynthesizing filter unit 66, selector 67, and control unit 68. That isto say, the wavelet inverse transformation device 51 has buffers (level3 buffer 61 through level 1 buffer 63) independent for each level otherthan the lowest band level (division level 4).

The division level 4 frequency components (4LH, 4HH) D61 and divisionlevel 4 frequency components (4LL, 4HL) D62 are input to the selector 64from an unshown external source (e.g., from a digital decoding unit 313shown in FIG. 28, which will be described later). The division level 3frequency components (3LH, 3HH) D63 and division level 3 frequencycomponents (3HL) D64 are input to the level 3 buffer 61. The divisionlevel 2 frequency components (2LH, 2HH) D65 and division level 2frequency components (2HL) D66 are input to the level 2 buffer 62. Thedivision level 1 frequency components (1LH, 1HH) D67 and division level1 frequency components (1HL) D68 are input to the level 1 buffer 63.

The level 3 buffer 61 stores and holds the coefficient (3LL) D80 of thedivision level 3 lowband components obtained as a result of horizontalsynthesizing filtering performed on the vertical synthesizing filteringresults of the division level 4, and the externally obtained divisionlevel 3 frequency components (3LH, 3HH) D63 and division level 3frequency components (3HL) D64. The division level 3 lowband componentcoefficient (3LL) D80 and the division level 3 frequency components(3HL) D64 are combined at the level 3 buffer 61, and output from thelevel 3 buffer 61 as division level 3 frequency components (3LL, 3HL)D70, with the division level 3 frequency components (3LH, 3HH) D63 beingoutput from the level 3 buffer 61 as division level 3 frequencycomponents (3LH, 3HH) D69 without change.

The level 2 buffer 62 stores and holds the division level 2 lowbandcomponents (2LL) D80 obtained as a result of horizontal synthesizingfiltering performed on the vertical synthesizing filtering results ofthe division level 3, and the externally obtained division level 2frequency components (2LH, 2HH) D65 and division level 2 frequencycomponents (2HL) D66. The division level 2 lowband components (2LL) D80and the frequency components (2HL) D66 are combined at the level 2buffer 62, and output from the level 2 buffer 62 as division level 2frequency components (2LL, 2HL) D72, with the division level 2 frequencycomponents (2LH, 2HH) D65 being output from the level 2 buffer 62 asdivision level 2 frequency components (2LH, 2HH) D71 without change.

The level 1 buffer 63 stores and holds the division level 1 lowbandcomponents (1LL) D80 obtained as a result of horizontal synthesizingfiltering performed on the vertical synthesizing filtering results ofthe division level 2, and the externally obtained division level 1frequency components (1LH, 1HH) D67 and division level 1 frequencycomponents (1HL) D68. The division level 1 lowband component coefficient(1LL) D80 and the frequency components (1HL) D68 are combined at thelevel 1 buffer 63, and output from the level 1 buffer 63 as divisionlevel 1 frequency components (1LL, 1HL) D74, with the division level 1frequency components (1LH, 1HH) D67 being output from the level 1 buffer63 as division level 1 frequency components (1LH, 2HH) D73 withoutchange.

Under control of the vertical synthesizing filter unit 65, the selector64 selects from an external source and from the level 3 buffer 61through level 1 buffer 63, the external source or the output of thecorresponding division level buffer, and outputs the selected output tothe vertical analysis filter unit 65 as frequency components (LH, HH)D75 and frequency components (LL, HL) D76.

The vertical synthesizing filter unit 65 refers to coefficients at thesame position in both the frequency components LL and frequencycomponents LH having the horizontal direction band L in common, collectsa predetermined number in the vertical direction (a number wherebyvertical synthesizing filtering can be executed), and performs verticalsynthesizing filtering. In the same way, the vertical synthesizingfilter unit 65 refers to coefficients at the same position in both thefrequency components HL and frequency components HL having thehorizontal direction band H in common, collects a predetermined numberin the vertical direction (a number whereby vertical synthesizingfiltering can be executed), and performs vertical synthesizingfiltering. The horizontal direction lowband (L) component D77 andhorizontal direction highband (H) component D78 generated as a result ofthe vertical synthesizing filtering are output to the horizontalsynthesizing filter unit 66.

The horizontal synthesizing filter unit 66 refers to coefficients at thesame position in the horizontal direction lowband (L) component D77 andhorizontal direction highband (H) component D78, collects apredetermined number in the horizontal direction (a number wherebyhorizontal synthesizing filtering can be executed), and performshorizontal synthesizing filtering. Consequently, lowband componentslines are generated in order from the top line of the image, and thegenerated lowband component (or image) D79 is output to the selector 67.

Under control of the control unit 68, in the event of proceeding to thenext division level the selector 67 outputs the lowband components D80to the buffer side of the level corresponding to the next divisionlevel, so as to store in the buffer of the level corresponding to thenext division level, and in the event that wavelet inversetransformation has ended as far as the initial division level in thewavelet transformation (i.e., the division level 1), the baseband imageD81 is externally output.

The control unit 68 is configured of a microcomputer or the likeincluding, for example, a CPU, ROM, and RAM, and controls the processingof the units of the wavelet inverse transformation device 51 byexecuting various types of programs.

Note that while the wavelet transformation device 1 shown in FIG. 1performs horizontal analysis filtering with Y and C interleaved, thewavelet inverse transformation device 51 shown in FIG. 18 performsprocessing with Y and C completely separated. That is to say, thewavelet inverse transformation device 51 actually has two separateconfigurations shown in FIG. 18, one each for Y and C.

The reason that the wavelet inverse transformation device 51 does notinterleave Y and C as with the wavelet transformation device 1 is thatthe vertical analysis filtering is first performed, following whichhorizontal analysis filtering is performed. That is to say, as describedabove with reference to FIG. 8, the vertical analysis filtering requiresan extra line buffer and also takes more processing time.

Next, the operations of the wavelet inverse transformation device 51shown in FIG. 18 will be described with reference to the flowchart inFIG. 19. That is to say, FIG. 19 illustrates the wavelet inversetransformation processing executed by the wavelet inverse transformationdevice 51, in which wavelet inverse transformation processing inversetransformation is performed from the lowband component with the smallestresolution in order to the highband components. In terms of divisionlevels, this is executed in the order of level 4, level 3, level 2, andlevel 1. Note that in FIG. 19, description is made without specifyingeither Y or C, but in reality, the processing of FIG. 19 is performedseparately and in parallel for Y and C.

In step S61, the selector 64 externally inputs division level 4frequency components (4LH, 4HH) D61 and division level 4 frequencycomponents (4LL, 4HL) D62. The selector 64 then selects, under controlof the vertical analyzing filter unit 65, output from this externalsource, and outputs the selected output (division level 4 frequencycomponents (4LH, 4HH) D61 and division level 4 frequency components(4LL, 4HL) D62) to the vertical analysis filter unit 65 as frequencycomponents (4LH, 4HH) D75 and frequency components (4LL, 4HL) D76.

In step S62, the vertical synthesizing filter unit 65 performs divisionlevel 4 vertical synthesizing filtering on the frequency components(4LH, 4HH) D75 and frequency components (4LL, 4HL) D76.

That is, as shown in FIG. 20, the vertical synthesizing filter unit 65references coefficients (indicted by circles in the drawing) at the sameposition for both the frequency components 4LL having a horizontaldirection band of L and vertical direction band of L, and frequencycomponents 4LH having a horizontal direction band of L and verticaldirection band of H (both having a horizontal direction band of L),collects a predetermined number in the vertical direction wherebyvertical synthesizing filtering can be performed (six in the case ofFIG. 20), and performs vertical synthesizing filtering.

Also, the vertical synthesizing filter unit 65 references coefficients(indicted by circles in the drawing) at the same position for both thefrequency components 4HL having a horizontal direction band of H andvertical direction band of L, and frequency components 4HH having ahorizontal direction band of H and vertical direction band of H (bothhaving a horizontal direction band of H), collects a predeterminednumber in the vertical direction whereby vertical synthesizing filteringcan be performed (six in the case of FIG. 20), and performs verticalsynthesizing filtering.

In the example shown in FIG. 20, an example is illustrated whereinvertical synthesizing filtering is performed using three coefficients ofthe frequency components 4LL and three coefficients at the same positionas the three coefficients of the frequency components 4LL at thefrequency components 4HL, and an example wherein vertical synthesizingfiltering is performed using the three coefficients of the frequencycomponents of 4HL and three coefficients of the frequency components 4HHat the same positions as the frequency components 4HL.

Consequently, horizontal direction lowband (L) components D77 andhorizontal direction highband (H) components D78 are generated, andoutput to the horizontal synthesizing filter unit 66.

In step S63, the horizontal synthesizing filter unit 66 performsdivision level 4 horizontal synthesizing filtering on the horizontaldirection lowband (L) components D77 and horizontal direction highband(H) components D78.

That is, as shown in FIG. 21, the horizontal synthesizing filter unit 66references coefficients (indicted by circles in the drawing) at the sameposition for the horizontal direction lowband (L) components D77 andhorizontal direction highband (H) components D78, collects apredetermined number in the horizontal direction whereby horizontalsynthesizing filtering can be performed (six in the case of FIG. 21),and performs horizontal synthesizing filtering.

In the example shown in FIG. 21, an example is illustrated whereinhorizontal synthesizing filtering is performed using three lowbandcomponent coefficients and three coefficients in the highband componentswhich are at the same position as the three lowband componentcoefficients.

Consequently, as shown in FIG. 22, a line of lowband components isgenerated in order from the top of the image, thereby generatingdivision level 3 lowband components (3LL) D79, output to the selector67. That is to say, the example in FIG. 22 shows a baseband image orlowband components of each level obtained as the result of lowbandcomponent lines being generated such that the first line of lowbandcomponents is generated in order from the top line of the image.

In step S64, the control unit 68 determines whether or not the waveletinverse transformation has proceeded to the division level 1, which isthe initial level for wavelet transformation (in other words, the finallevel for wavelet inverse transformation), and in the event thatdetermination is made that the division level 1 is not ended, theprocessing advances to step S65.

In step S65, the control unit 68 controls the selector 67 so as toselect output to the level 3 buffer 61, and stores the division level 3lowband components (3LL) D80 obtained as the result of the horizontalsynthesizing filtering, in the corresponding level buffer (in this case,the level 3 buffer 61).

In step S66, the control unit 68 transfers the division level 3frequency components (3LH, 3HH) D63 from the external source and thedivision level 3 frequency components (3HL) D64 to the level 3 buffer61, so as to be stored.

In step S67, the vertical analysis filter unit 65 controls the selector64 so as to select output from the level 3 buffer 61, thereby readingout frequency components from the level 3 buffer 61, and the frequencycomponents that have been read out are output to the vertical analysisfilter unit 65 as frequency components (3LH, 3HH) D75 and frequencycomponents (3LL, 3HL) D76.

That is to say, at the level 3 buffer 61, the division level 3 lowbandcomponents (3LL) D80 and the division level 3 frequency components (3HL)D64 are combined and output from the level 3 buffer 61 as division level3 frequency components (3LL, 3HL) D70, and the division level 3frequency components (3LH, 3HH) D63 are output from the level 3 buffer61 as division level 3 frequency components (3LH, 3HH) D69, withoutchange. Accordingly, division level 3 frequency components (3LL, 3HL)D70, and division level 3 frequency components (3LH, 3HH) D69, are eachoutput to the vertical analysis filter unit 65 as frequency components(3LH, 3HH) D75 and frequency components (3LL, 3HL) D76.

Subsequently, the processing returns to step S62, and subsequentprocessing is repeated. That is to say, in step S62, division level 3vertical synthesizing filtering is performed, in step S63, divisionlevel 3 horizontal synthesizing filtering is performed, and divisionlevel 2 lowband components (2LL) are generated.

In this case, in step S64 determination is made that the division level1 has not yet ended, so the division level 2 lowband component (2LL) D80obtained as the result of horizontal analysis filtering is stored in thecorresponding level buffer (in this case, the level 2 buffer 62). In thesame way, the division level 2 frequency components (2LH, 2HH) D65 anddivision level 2 frequency components (2HL) D66 from the external sourceare transferred to the level 3 buffer 61.

At this time, in the same way as with the case of the level 3 buffer 61,at the level 2 buffer 62 the division level 2 lowband components (2LL)D80 and the division level 2 frequency components (2HL) D64 arecombined, and output from the level 2 buffer 62 as division level 2frequency components (2LL, 2HL) D70, while the division level 2frequency components (2LH, 2HH) D63 are output from the level 2 buffer62 as division level 2 frequency components (2LH, 2HH) D69, withoutchange.

The above series of processing is performed until the division level 1frequency components are stored in the level 1 buffer 63 and read out.Subsequently, in step S62, division level 1 vertical synthesizingfiltering is performed, and at step S63, division level 1 horizontalsynthesizing filtering is performed. Consequently, a baseband imagewherein synthesizing filtering has ended to division level 1 isgenerated, and in step S64, determination is made that through divisionlevel 1 has ended, so the flow proceeds to step S68, and in step S68 thebaseband image D81 from the horizontal synthesizing filter unit 66 isoutput externally (e.g., to a later-described inverse quantization unit162 shown in FIG. 26), via the selector 67.

As described above, the wavelet inverse transformation device 51 shownin FIG. 18 also is configured to be able to handle buffers for eachdivision level other than the lowest band level with internal memory, sothe horizontal synthesizing filtering results can be stored in buffersof each division level while performing the horizontal synthesizingfiltering. Accordingly, vertical synthesizing filtering can be performedwhile reading out the results of the horizontal synthesizing filteringfrom the buffer of each division level. That is to say, horizontaldirection and vertical direction filtering can be performedsimultaneously in parallel.

Thus, wavelet transformation can be performed at high speed for movingimages and images with high resolution, as well.

Also, internal memory handles the buffers for each division level otherthan the lowest band level, so there is no need to configure externalmemory as with known arrangements.

Accordingly, there is no need to exchange data with external memory, andwavelet inverse transformation can be performed at high speeds.Consequently, there is no need to raise the clock frequency in order toincrease speed of data between the external memory and the waveletinverse transformation device, thereby conserving electric power.

Further, with the wavelet inverse transformation device, Y and C areprocessed completely separately since vertical synthesizing filtering isperformed before the horizontal synthesizing filtering, and accordinglyas described above with reference to FIG. 8, necessity of an extra linebuffer at the vertical synthesizing filtering is prevented, and alsoexcess processing time can be suppressed.

While already described above with reference to FIG. 11 for the case ofthe wavelet transformation device, effective filtering can be performedfor analyzing filtering by using the lifting technique. Accordingly, thelifting technique can be similarly used with the synthesizing filteringfor wavelet inverse transformation as well.

FIG. 23 illustrates a lifting scheme of a 5×3 analysis filter used withthe JPEG (Joint Photographic Experts Group) 2000 standard as well.Synthesizing lifting in a case of applying the lifting technique to this5×3 analysis filter will be described.

In the example shown in FIG. 23, the top portion represents coefficientsgenerated by wavelet transformation, with the solid circles representinghighband component coefficients, and the solid squares representinglowband component coefficients.

As the first stage, even-numbered (starting from 0) coefficients aregenerated from the input lowband component and highband componentcoefficients, as shown in the following Expression (4).even-numbered coefficient s _(i) ⁰ =s _(i) ¹−1/4(d _(i-1) ¹ +d _(i)¹)  (4)

Next, as the second stage, odd-numbered (starting from 0) coefficientsare generated from the even-numbered coefficients generated at the firststage and the input highband component coefficients, as shown in thefollowing Expression (5).odd-numbered coefficient d _(i) ⁰ =d _(i) ¹+1/2(s _(i) ⁰ +is ₊₁ ⁰)  (5)

Thus, with synthesizing filtering, first the even-numbered coefficientsare generated, following which odd-numbered coefficients are generated.The two types of filter banks used for the synthesizing filtering are oftwo taps in the same way as with that described above with FIG. 11,which is far shorter than the originally-necessary five taps, therebymarkedly reducing the amount of calculations.

Also, while the above description has been made with reference to FIGS.12 through 16 regarding an example of performing wavelet transformationon video signals which are moving images, there is also the need toperform wavelet inverse transformation at high speeds in cases ofperforming wavelet inverse transformation of frequency componentcoefficients generated (divided) by wavelet transformation in incrementsof pictures making up the video signals, as well.

Accordingly, as with the case of the Y vertical analysis filter unit 17and C vertical analysis filter unit 18 in the wavelet transformationdevice 1, the vertical synthesizing filter unit 65 of the waveletinverse transformation device 51 also performs vertical directionsynthesizing filtering immediately at the point that a predeterminednumber of frequency component coefficients are accumulated in thevertical direction (i.e., as many as are necessary for executing thevertical synthesizing filtering).

Moreover, as with the case of the horizontal analysis filter unit 11 inthe wavelet transformation device 1, the horizontal synthesizing filterunit 66 of the wavelet inverse transformation device 51 also performshorizontal direction synthesizing filtering immediately at the pointthat a predetermined number of frequency component coefficients areaccumulated in the horizontal direction (i.e., as many as are necessaryfor executing the horizontal synthesizing filtering).

As described above, synthesizing filtering is performed as soon as apredetermined number of frequency component coefficients are accumulatedin the vertical direction and in the horizontal direction, sosynthesizing filtering of one picture of video signals can beeffectively performed. That is to say, wavelet inverse transformationcan be performed at high speed.

Further, description has been made above with reference to FIG. 17regarding a case wherein, in the event of performing wavelettransformation of video signals which are moving images, with thewavelet transformation device 1 shown in FIG. 1, the end of a picture isdetected by having an arrangement for detecting vertical synchronizingsignals of video signals.

In the event of detecting the end of a picture as described in FIG. 17,and performing wavelet inverse transformation of frequency componentsgenerated by wavelet transformation in increments of pictures making upthe video signals, an arrangement not shown in the drawings is provideddownstream of the selector 67 of the wavelet inverse transformationdevice 51, as a vertical synchronizing signal insertion portion forinserting video vertical synchronizing signals after picture signalsgenerated by wavelet inverse transformation (i.e., the above-describedbaseband image D81).

That is to say, a vertical synchronizing signal insertion portion isprovided downstream of the selector 67 of the wavelet inversetransformation device 51, so as to insert video vertical synchronizingsignals after picture signals, e.g., after the baseband image D81 fromthe selector 67, and the generated video signals are externally output.

Thus, continuously inserting video vertical synchronizing signals forsubsequent pictures as well, enables generated video signals to besequentially output. Accordingly, moving images can be reproduced.

As described above, with the wavelet transformation device of anembodiment of the invention, buffers are provided for each divisionlevel from level 1 to a predetermined number of levels, and thehorizontal analysis filtering results are stored in each division levelbuffer while performing the horizontal analysis filtering, so verticaldirection filtering can be performed while reading out the results ofthe horizontal analysis filtering from the buffer of each divisionlevel. That is to say, horizontal direction and vertical directionfiltering can be performed simultaneously in parallel. That is to say,horizontal direction and vertical direction filtering can be performedsimultaneously in parallel. Thus, wavelet transformation can beperformed at high speed for moving images and images with highresolution, as well.

Also, internal memory handles the buffers for each division level fromlevel 1 to a predetermined number of levels, so there is no need toexchange data with external memory, and wavelet transformation can beperformed at high speeds. Consequently, there is no need to raise theclock frequency in order to increase speed of data between the externalmemory and the wavelet inverse transformation device, thereby conservingelectric power.

Also, horizontal analysis filtering is performed on frequency componentswherein Y and C have been interleaved, so a configuration can be madewith just one horizontal analysis filter unit, which is a markedcontribution to reduction in the scale of hardware.

Further, the highband components and lowband components which are theresults of the horizontal analysis filtering are interleaved for each ofY and C, and stored in buffers of corresponding levels, Y and C beingstored separately, so at the time of reading out, all that is necessaryis to read out from the front of the buffer of that level, therebysimplifying control.

Also, vertical direction analysis filtering is performed separately forY and C, so there is no need for the massive memory capacity which isnecessary in the event of not performing the vertical direction analysisfiltering separately for Y and C for example, and drastic increases incost can be prevented. Further, the need for extra processing time canbe prevented, as well.

On the other hand, with the wavelet inverse transformation device of anembodiment of the invention, buffers are provided for each divisionlevel except for the lowest level, and the horizontal synthesizingfiltering results are stored in each division level buffer whileperforming the horizontal synthesizing filtering, so vertical directionfiltering can be performed while reading out the results of thehorizontal synthesizing filtering from the buffer of each divisionlevel. That is to say, horizontal direction and vertical directionfiltering can be performed simultaneously in parallel. Thus, wavelettransformation can be performed at high speed for moving images andimages with high resolution, as well.

Also, internal memory handles the buffers for each division level otherthan the lowest band level, so there is no need to configure externalmemory and there is no need to exchange data with external memory, andwavelet inverse transformation can be performed at high speeds.Consequently, there is no need to raise the clock frequency in order toincrease speed of data between the external memory and the waveletinverse transformation device, thereby conserving electric power.

Also, unlike wavelet transformation, Y and C are processed completelyseparately without interleaving Y and C, thereby preventing extra memorybeing necessary for C at the time of vertical synthesizing filtering,and also excess processing time can be suppressed.

Further, with the wavelet transformation device and wavelet inversetransformation device according to an embodiment of the presentinvention, analysis filtering and synthesizing filtering is performed assoon as a predetermined number of frequency component coefficients areaccumulated, so analysis filtering and synthesizing filtering can beeffectively performed. That is to say, wavelet transformation andwavelet inverse transformation can be performed at high speed, so as tobe capable of handling wavelet transformation and wavelet inversetransformation of video signals input at 60 fields per second, which iseach picture being input at 1/60 seconds.

Thus, parallel processing per line is enabled in a later-describedtransmission system including encoding processing using wavelettransformation and decoding processing using wavelet inversetransformation, thereby obtaining a decoded image with little delay.

Also, a vertical synchronizing signal detecting arrangement is providedto the wavelet transformation device according to an embodiment of thepresent invention, and a vertical synchronizing signal insertionarrangement is provided to the wavelet inverse transformation deviceaccording to an embodiment of the present invention, so analysisfiltering can be performed on video signals in increments of pictures(fields or frames) making up the video signals.

The embodiment of the present invention as described above relates to adevice or method for performing wavelet transformation of images orvideo signals, and also relates to a device or method for performingwavelet inverse transformation wherein synthesizing filtering of banddivided information is performed so as to restore into images or videosignals. Various applications can be conceived for such a device ormethod.

That is to say, description has been made above regarding the wavelettransformation device 1 which performs wavelet transformation of imagesof video signals to divide image signals and video signals into multiplefrequency components, and also a wavelet inverse transformation device51 for performing wavelet inverse transformation of the frequencycomponents generated by the wavelet transformation device 1, but wavelettransformation is widely used as pre-processing for image compression.Accordingly, description will now be made regarding an image encodingdevice for performing compression encoding of frequency componentsgenerated by wavelet transformation (hereafter also referred to as“coefficient data”), and an image decoding device for decoding thecoefficient data subjected to compression encoding by the image encodingdevice.

FIG. 24 is a diagram illustrating a configuration example according toan embodiment of an image encoding device to which an embodiment of thepresent invention has been applied. With this image encoding device,wavelet transformation according to an embodiment of the presentinvention is performed as pre-processing for compression.

In the example shown in FIG. 24, the image encoding device 101 isconfigured of a wavelet transformation unit 111, a quantization unit112, an entropy encoding unit 113, and a rate control unit 114.

The wavelet transformation unit 111 is configured basically in the sameway as the wavelet transformation device 1 shown in FIG. 1. That is tosay, the wavelet transformation device 111 has independent buffers foreach division level (level 1 buffer 12 through level 4 buffer 15),wherein, at the point that input video signals D110 (equivalent to imagesignals D12 wherein Y and C are interleaved) being accumulated to apredetermined number of columns, horizontal analysis filtering isimmediately performed on the video signals D110, and the coefficientdata (frequency components) obtained as a result of the horizontalanalysis filtering is stored in the buffers corresponding to each level.Upon the coefficient data which has been obtained as a result of thehorizontal analysis filtering being accumulated to a predeterminednumber of lines in the buffers corresponding to each level, verticalanalysis filtering is performed separately for Y and C as to thecoefficient data, which is repeated to the predetermined division level,and the post-analysis coefficient data D111 is supplied to thequantization unit 112.

For example, with the division level 2 analysis filtering, as shown inFIG. 16, wavelet transformation is performed of the four lines of 1LLgenerated by division level 1 analysis filtering, thereby yielding thetwo lines of 2LL, 2HL, 2LH, and 2HH. At the division level 3 analysisfiltering, two lines of 2LL are subjected to wavelet transformation,thereby yielding one line of 3LL, 3HL, 3LH, and 3HH. In the event thatthe division level 3 is the final analysis filtering, 3LL is the lowestband.

Note that as described above with reference to FIG. 17, when inputtingvideo signals, detecting vertical synchronizing signals (i.e., pictureend) in the video signals stops the analysis filtering operations at theend of the picture, with wavelet transformation being performed for eachpicture.

The quantization unit 112 quantizes the coefficient data D111 generatedby the wavelet transformation unit 111, by dividing by a quantizationstep size for example, thereby generating quantized coefficient dataD112.

At this time, the quantization unit 112 takes, as a line block, anincrement configured of one line worth of the lowest band frequencycomponent generated (3LL in the case in FIG. 16) and multiple lines ofother frequency components necessary for generating that one line, andcan set the quantization step size for each such line block. This lineblock comprehensively includes the coefficients of all frequencycomponents for a certain image region (in the case of FIG. 16, the 10frequency components of 3LL through 1HH), so performing quantization foreach line block enables the feature of wavelet transformation, which isthe advantage of multiple resolution analysis, to be utilized. Also,only the number of line blocks needs to be determined for the entirescreen, reducing the load on the image encoding device 101.

Further, energy of image signals generally is concentrated at thelowband components, and also, deterioration in lowband components tendsto be more conspicuous to human visual perception, so quantization canbe advantageously weighted such that the quantization step sizes oflowband component sub-bands are ultimately smaller. This weightingappropriates a relatively greater amount of information to the lowbandcomments, consequently improving the overall impression of imagequality.

The entropy encoding unit 113 performs source coding on the quantizedcoefficient data D112 generated at the quantization unit 112, therebygenerating compressed encoded code stream data D113. As for sourcecoding, Huffman coding used with JPEG or MPEG (Moving Picture ExpertsGroup), or even higher-efficiency arithmetic coding used with JPEG 2000,can be used.

Now, coefficients of which range to apply the entropy encoding to is anextremely important issue, directly related to compression efficiency.With the JPEG and MPEG methods for example, DCT (Discrete CosineTransform) is performed on blocks of 8×8, and Huffman encoding isperformed on the generated 64 DCT coefficients, thereby compressing theinformation. That is to say, the 64 DCT coefficients is the range ofentropy encoding.

With the wavelet transformation unit 111, wavelet transformation isperformed in increments of lines, unlike DCT which is performed onblocks of 8×8, so at the entropy encoding unit 113, source coding isperformed independently for each frequency band, and for each P linewithin each frequency band.

One line is the minimum for P, but the fewer number of lines, the lessreference information is required, meaning that the memory capacity canbe reduced that much. Conversely, the more lines there are, the moreinformation amount there is accordingly, so encoding efficiency can beimproved. However, in the event that P exceeds the number of lines ofthe line block within the frequency bands, this will require lines ofthe next line block. Accordingly, the processing will need to wait forquantization coefficient data for this line block to be generated bywavelet transformation and quantization, and this wait time will becomedelay time.

Accordingly, if reducing delay time is desired, there is the need tokeep P within the number of lines of the line block. For example, in thecase shown in FIG. 16, for the frequency bands of 3LL, 3HL, 3LH, and3LL, the number of lines of the line blocks is 1, so P=1. Also, for thesub-bands of 2HL, 2LH, and 2HH, the number of lines of the line blocksis 2, so P=1 or 2.

The rate control unit 114 performs control for ultimately matching thetarget bit rate or compression rate, and externally outputs thepost-rate-control encoded code stream D114. For example, the ratecontrol unit 114 transmits control signals D115 to the quantization unit112 so as to reduce the quantization step size in the event of raisingthe bit rate, and increase the quantization step size in the event oflowering the bit rate.

Next, the image encoding processing of the image encoding device 101shown in FIG. 24 will be described with reference to the flowchart shownin FIG. 25.

Video signals D110 are input to the wavelet transformation unit 111externally (e.g., from a later-described video camera unit 303 shown inFIG. 28). In step S111, the wavelet transformation unit 111 performswavelet transformation processing on the image signals D110. Note thatthis wavelet transformation processing is processing which is performedfor each picture from which a vertical synchronizing signal is detectedform the video signals D110, in increments of lines, but the processingis generally the same as the wavelet transformation processing describedabove with reference to FIG. 2, so description thereof will be omitted.

With the wavelet transformation processing of step S111, at the pointthat input video signals D110 (equivalent to image signals D12 wherein Yand C have been interleaved) are accumulated to a predetermined numberof columns, horizontal analysis filtering is immediately performed onthe video signals D110, and the coefficient data (frequency components)obtained as a result of the horizontal analysis filtering is stored inthe buffers corresponding to each level. Upon the coefficient data whichhas been obtained as a result of the horizontal analysis filtering beingaccumulated to a predetermined number of lines of the bufferscorresponding to each level, vertical analysis filtering is immediatelyperformed separately for Y and C as to the coefficient data, which isrepeated to a predetermined division level, and the post-analysiscoefficient data D111 is supplied to the quantization unit 112.

That is to say, as described above with reference to FIGS. 13 through16, the wavelet transformation unit 111 performs filtering processingwhereby coefficient data of one line of the lowest band can be obtained,multiple times in stages for all lines of the entire screen.

In step S112, the quantization unit 112 quantizes the coefficient dataD111 generated by the wavelet transformation unit 111, by dividing by aquantization step size for example, thereby generating quantizedcoefficient data D112.

At this time, the quantization unit 112 takes, as a line block, anincrement configured of one line worth of the lowest band frequencycomponent generated (3LL in the case in FIG. 16) and multiple lines ofother frequency components necessary for generating that one line, andsets the quantization step size for each such line block. That is tosay, upon the predetermined number of lines being accumulated, thequantization unit 112 also performs quantization immediately, for eachline block.

In step S113, the entropy coding unit 113 performs entropy encoding(source coding) of the quantization coefficient data D112 generated atthe quantization unit 112, and generates a compressed encoded codestream D113.

Now, at the wavelet transformation unit 111, wavelet transformation isperformed in increments of lines, so the entropy encoding unit 113 alsoperforms source coding independently for each frequency band, and foreach P line within each frequency band. That is to say, upon P lines(within the number of lines in a line block) being accumulated, theentropy encoding unit 113 also performs source coding immediately, foreach line block.

In step S114, the rate control unit 114 performs rate control (i.e.,control for ultimately matching the target bit rate or compression rate)and externally outputs the post-rate-control encoded code stream D114.

As described above, with the image encoding device, wavelettransformation is performed in increments of lines, quantization isperformed in increments of line blocks, and source coding is performedfor every P lines which is a number within the number of lines in a lineblock, and the encoded code stream D114 which has been encoded for eachP line, is externally output. That is to say, wavelet transformationprocessing, quantization processing, and source coding processing, canbe operated in parallel in predetermined increments of lines.

Accordingly, in the event that encoded data encoded by the informationencoding device is transmitted for example, data encoded every P linesis sequentially transmitted, so a decoded image can be obtained at theimage decoding device which receives and decodes the encoded data (imagedecoding device 151 in FIG. 26), with little delay.

FIG. 26 is a diagram illustrating a configuration example according toan embodiment of an image decoding device shown in FIG. 24,corresponding to the image encoding device.

In the example shown in FIG. 26, the image decoding device 151 isconfigured of an entropy decoding unit 161, an inverse quantization unit162, and a wavelet inverse transformation unit 163.

The entropy decoding unit 161 performs source decoding on the inputencoded code stream D160, and generates quantized coefficient data D161.As for source decoding, Huffman coding, or even higher-efficiencyarithmetic coding, or the like can be used corresponding to the sourcecoding performed by the image encoding device 101. In the event thatsource coding has been performed at the image encoding device 101independently for each P line, as described above with FIG. 24, theentropy decoding unit 161 also performs source decoding independentlyfor each frequency band, and for each P line within each frequency band.

The inverse quantization unit 162 performs inverse quantization bymultiplying the quantized coefficient data D161 by the quantization stepsize, thereby generating coefficient data D162. This quantization stepsize is normally described in the header of the encoded code stream orthe like. Note that, in the event that the quantization step size is setat the image encoding device 101 for each line block as described abovewith reference to FIG. 24, the inverse quantization unit 162correspondingly sets the inverse quantization step size for each lineblock, and performs inverse quantization.

The wavelet inverse transformation unit 163 is configured basically inthe same way as the wavelet inverse transformation device 51 shown inFIG. 18. That is to say, the wavelet inverse transformation device 163has independent buffers for each division level (level 3 buffer 61through level 1 buffer 63) other than the lowest band level, whereinvertical synthesizing filtering and horizontal synthesizing filtering isperformed on the coefficient data D162, and coefficient data obtained asa result of the horizontal analysis filtering is stored in the bufferscorresponding to each level. Upon the coefficient data being accumulatedto a predetermined number in the buffers corresponding to each level,the vertical synthesizing filtering and horizontal synthesizingfiltering is immediately performed, which is repeated to level 1,thereby generating the baseband image. Further, the wavelet inversetransformation device 163 inserts vertical synchronizing signals in thebaseband image so as to generate video signals D163, which areexternally output.

Next, the image decoding processing of the image decoding device 151shown in FIG. 26 will be described with reference to the flowchart shownin FIG. 27.

The entropy decoding unit 161 has input thereto an encoded code streamD160 that has been encoded by the image encoding processing describedabove with reference to FIG. 25, from an external source (e.g., from adigital decoding unit 313 shown in FIG. 28, which will be describedlater). In step S161, the entropy decoding unit 161 performs entropydecoding (source decoding) of the input encoded code stream D160,thereby generating quantized coefficient data D161.

At this time, source coding has been performed for each P line at theimage encoding unit 101, so the entropy decoding unit 161 also performssource decoding independently for each frequency band, and for each Pline within each sub-band.

The inverse quantization unit 162 performs inverse quantization bymultiplying the quantized coefficient data D161 by the quantization stepsize, thereby generating coefficient data D162.

Now, the quantization step size is set at the image encoding device 101for each line block, so the inverse quantization unit 162correspondingly sets the inverse quantization step size for each lineblock, and performs inverse quantization.

In step S163, the wavelet inverse transformation device 163 performswavelet inverse transformation processing on the coefficient data D162.Note that this wavelet inverse transformation processing is processingwhich is performed in increments of lines, with vertical synchronizingsignals being inserted following the image being generated, but theprocessing is generally the same as the wavelet inverse transformationprocessing described above with reference to FIG. 19, so descriptionthereof will be omitted.

With the wavelet inverse transformation processing in step S163,vertical synthesizing filtering and horizontal synthesizing filtering isperformed on the coefficient data D162, and coefficient data obtained asa result of the horizontal synthesizing filtering is stored in thebuffers corresponding to each level, wherein, upon the coefficient datastored in the buffers corresponding to each level being accumulated to apredetermined number, vertical synthesizing filtering and horizontalsynthesizing filtering is immediately performed, which is repeated tolevel 1, thereby generating the baseband image. Further, verticalsynchronizing signals are inserted in the generated baseband image so asto generate video signals D163, which are externally output (e.g., to avideo camera unit 303 shown in FIG. 28, which will be described later).

That is to say, the image encoding device 101 performs wavelettransformation processing in increments of lines, so in the same way,the image decoding device 163 performs wavelet inverse transformationprocessing in increments of lines.

As described above, with the image decoding device 151, the inputencoded code stream is subjected to source decoding for each P line,inverse quantization is performed in increments of line blocks, andwavelet inverse transformation is performed in increments of lines,thereby generating the baseband image. Vertical synchronizing signalsare further inserted in the baseband image so as to generate videosignals D163, which are externally output. That is to say, decodingprocessing, inverse quantization processing, and wavelet inversetransformation processing can be operated in parallel in predeterminedincrements of lines.

Accordingly, in the event that encoded data is transmitted, encoded datawhich is sequentially transmitted is decoded every P lines and generatedin increments of lines, so a decoded image can be obtained with littledelay.

As described above, the processing of each of the image encoding device101 and image decoding device 151 described with reference to FIGS. 24and 26 can be operated in parallel in increments of lines, whereby imagecompression encoding and decoding processing can be performed with lessdelay.

Next, examples of applying the image encoding device 101 and imagedecoding device 151 described with reference to FIGS. 24 and 26 tovarious systems will be described.

FIG. 28 illustrates the configuration of an example of a digital triaxsystem to which the image encoding including the wavelet transformation,and image decoding including the wavelet inverse transformation,according to an embodiment of the present invention, can be applied.

A triax system is a system used in television broadcasting stations,production studios, and so forth. With such a system, at the time ofrecording in the studio or broadcasting live from a remote location, asingle triaxial cable connecting a video camera and a camera controlunit or a switcher is used to transmit multiplex signals such as picturesignals, audio signals, return picture signals, synchronizing signals,and so forth, and also to supply power.

Many known triax systems have been arranged to transmit theabove-described signals in the form of analog signals. On the otherhand, in recent years, entire systems are becoming digital, andaccordingly, triax systems used in television broadcasting stations arealso becoming digital.

With known digital triax systems, the digital video signals transmittedover the triax cable have been uncompressed video signals. The reasonfor this is that the specs demanded regarding signal delay time areparticularly severe with television broadcasting stations; basically,the delay time from shooting to monitor output, for example, is requiredto be within one field (16.67 msec). Compression encoding systems suchas MPEG2 and MPEG4 which have realized high compression rates and highimage quality have not been used in triax systems since time equivalentto several frames worth is required for video signal compression andencoding, and decoding of compressed video signals, meaning that delaytime is great.

Image encoding including the wavelet transformation, and image decodingincluding the wavelet inverse transformation, according to an embodimentof the present invention, is capable of parallel operations for thehorizontal filtering and vertical filtering as described above withreference to FIGS. 2 and 19, and also is capable of parallel operationsdue to operations being performed in increments of lines, as describedabove with reference to FIGS. 25 and 27; accordingly, the delay timefrom image data input till obtaining of an output image can be reduced,such that application can be made to a digital triax system.

The digital triax system shown in FIG. 28 is configured with atransmission unit 300 and camera control unit 302 connected via a triaxcable (triaxial cable) 301. Digital video signals and digital audiosignals (hereafter referred to as “main line signals”) from thetransmission unit 300 to the camera control unit 302 which are actuallybroadcast, or used as contents, and intercom audio signals and returndigital video signals from the camera control unit 302 to thetransmission unit 300, are transmitted over the triax cable 301.

The transmission unit 300 is built into an unshown video camera device,for example. Of course, other arrangements may be made, such as thetransmission unit 300 being connected to the video camera device as anexternal device of the video camera device. The camera control unit 302may be a device commonly called a CCU (Camera Control Unit), forexample.

Digital audio signals have little bearing on the essence of the presentinvention, so description thereof will be omitted for the sake ofsimplicity in description.

The video camera unit 303 is configured within an unshown video cameradevice for example, and performs photoreception with an unshownimage-taking device such as a CCD (Charge Coupled Device), of light froma subject that has been taken in via an optical system 350 including alens, focusing mechanism, zooming mechanism, iris adjusting mechanism,and so forth. The image-taking device converts the received light intoelectrical signals by photoelectric conversion, and further performspredetermined signals processing, so as to output as baseband digitalvideo signals. These digital video signals are mapped to an HD-SDI (HighDefinition Serial Data Interface) format for example, and output.

Also connected to the video camera unit 303 are a display unit 351 usedas a monitor, and an intercom 352 used for exchanging audio externally.

The transmission unit 300 has a video signal encoding unit 310 and videosignal decoding unit 311, digital modulation unit 312 and digitaldemodulation unit 313, amplifiers 314 and 315, and a videosplitting/synthesizing unit 316.

Baseband digital video signals mapped to the HD-SDI format for example,and supplied from the video camera unit 303 to the transmission unit300. The digital video signals are compressed and encoded at the videosignal encoding unit 310 so as to become a code stream, which issupplied to the digital modulation unit 312. The digital modulation unit312 modulates the supplied code stream into a format suitable fortransmission over the triax cable 301, and outputs. The signals outputfrom the digital modulation unit 312 are supplied to the videosplitting/synthesizing unit 316 via an amplifier 314. The videosplitting/synthesizing unit 316 sends the supplied signals to the triaxcable 301. These signals are received at the camera control unit 302 viathe triax cable 302.

The signals output from the camera control unit 302 are received at thetransmission unit 300 via the triax cable 301. The received signals aresupplied to the video splitting/synthesizing unit 316, and the portionof digital video signals and the portion of other signals are separated.Of the received signals, the portion of the digital video signals issupplied via an amplifier 315 to the digital demodulation unit 313, thesignals modulated into a format suitable of transmission over the triaxcable 301 are demodulated at the camera control unit 302 side, and thecode stream is restored.

The code stream is supplied to the video signal decoding unit 311, thecompression coding is decoded, and the baseband digital video signalsare obtained. The decoded digital video signals are mapped to the HD-SDIformat and output, and supplied to the video camera unit 303 as returndigital video signals. The return digital video signals are supplied tothe display unit 351 connected to the video camera unit 303, and usedfor monitoring by the camera operator.

The cameral control unit 302 has a video splitting/synthesizing unit320, amplifiers 321 and 322, a front-end unit 323, a digitaldemodulation unit 324 and digital modulation unit 325, and a videosignal decoding unit 326 and video signal encoding unit 327.

Signals output from the transmission unit 300 are received at the cameracontrol unit 302 via the triax cable 301. The received signals aresupplied to the video splitting/synthesizing unit 320. The videosplitting/synthesizing unit 320 supplies the signals supplied thereto tothe digital demodulation unit 324 via the amplifier 321 and front endunit 323. Note that the front end unit 323 has a gain control unit foradjusting gain of input signals, a filter unit for performingpredetermined filtering on input signals, and so forth.

The digital demodulation unit 324 demodulates the signals modulated intoa format suitable of transmission over the triax cable 301 at thetransmission unit 300 side, and restores the code stream. The codestream is supplied to the video signal decoding unit 326 wherecompression encoding is decoded, so as to obtain the baseband digitalvideo signals. The decoded digital video signals are mapped to theHD-SDI format and output, and externally output as main line signals.

The return digital video signals and digital audio signals are suppliedexternally to the camera control unit 302. The digital audio signals aresupplied to the intercom 352 of the camera operator for example, to beused for transmitting external audio instructions to the cameraoperator.

The return digital video signals are supplied to the video signalencoding unit 327 and compression encoded, and supplied to the digitalmodulation unit 325. The digital modulation unit 325 modulates thesupplied code stream into a format suitable for transmission over thetriax cable 301, and outputs. The signals output from the digitalmodulation unit 325 are supplied to the video splitting/synthesizingunit 320 via the front end unit 323 and amplifier 322. The videosplitting/synthesizing unit 320 multiplexes these signals with othersignals, and sends out to the triax cable 301. The signals are receivedat the transmission unit 300 via the triax cable 301.

In the example shown in FIG. 28, the image encoding device 101 in FIG.24 and the image decoding device 151 shown in FIG. 26 are respectivelyapplied to the video signal encoding units 310 and 327, and the videosignal decoding units 311 and 326. That is to say, the video signalencoding units 310 and 327 are basically configured the same as with theimage encoding device 101 in FIG. 24, and video signal decoding units311 and 326 are basically configured the same as with the image decodingdevice 151 in FIG. 26.

That is to say, at the transmission unit 300 side, the video signalencoding unit 310 performs the wavelet transformation and entropyencoding described above with reference to FIG. 25, on the digital videosignals supplied thereto, and outputs a code stream. As described abovewith reference to FIGS. 13 through 16, upon a number of linescorresponding to the number of taps of the filter used for wavelettransformation and according to the number of division levels of wavelettransformation being input, the video signal encoding unit 310 startswavelet transformation. Further, as described above with reference toFIGS. 25 and 27, upon coefficient data necessary for the componentsbeing accumulated at the image encoding device and image decodingdevice, processing is sequentially performed by the components. Uponprocessing ending to the bottom line of one frame or one field,processing of the next one frame or one field is started.

As described above, with the image encoding device 101 and imagedecoding device 151 shown in FIGS. 24 and 26, the components thereofperform processing in parallel, so the image encoding device 101 andimage decoding device 151 can suppress delay of output of pictures takenby the video camera unit 303 from the camera control unit 302, and delayof return digital video signals supplied externally and transmitted fromthe cameral control unit 302 to the video camera unit 303, andaccordingly are advantageously used in the digital triax system shown inFIG. 27.

This also holds true for transmitting return digital video signals fromthe camera control unit 302 side to the transmission unit 300 side. Thatis to say, the above-described wavelet transformation and entropyencoding in FIG. 25 is performed on the externally supplied returndigital video signals by the video signal encoding unit 327, and a codestream is output.

Now, there are many cases wherein it is permissible for the returndigital video signals to be of a lower image quality than the digitalvideo signals of the main line signals. In this case, the bit rate atthe time of encoding at the video signal encoding unit 327 can belowered.

For example, the video signal encoding unit 327 performs control withthe rate control unit 114 such that the bit rate of entropy encodingprocessing at the entropy encoding unit 113 is lower. Also, anarrangement can be conceived, wherein, for example, at the cameracontrol unit 302 side, transformation processing is performed to ahigher division level with the wavelet transformation unit 111 at thevideo signal encoding unit 327, and at the transmission unit 300 side,the wavelet inverse transformation at the wavelet inverse transformationunit 163 of the video signals encoding unit 311 is stopped at a lowerdivision level. Processing at the video signal encoding unit 327 of thecamera control unit 302 is not restricted to this example; and variousother types of processing can be conceived, such as keeping the divisionlevel for wavelet transformation low so as to alleviate the load oftransformation processing.

FIG. 29 illustrates the configuration of an example of a wirelesstransmission system to which the image encoding including the wavelettransformation, and image decoding including the wavelet inversetransformation, according to an embodiment of the present invention, canbe applied. That is to say, with the example shown in FIG. 29,transmission of coded data encoded at the image encoding deviceincluding the wavelet transformation according to an embodiment of thepresent invention, to an image decoding device side, is performedwirelessly.

Note that in the example in FIG. 29, video signals are transmittedunidirectionally from the video camera or transmission unit 400 side(hereafter abbreviated as “transmission unit 400”) to the receptiondevice 401 side. Bidirectional communication between the transmissionunit 400 and reception unit 401 can be performed for audio signals andother signals.

The transmission unit 400 is built into an unshown video camera devicehaving a video camera unit 402, for example. Of course, otherarrangements may be made, such as the transmission unit 400 beingconnected to the video camera device as an external device of the videocamera device having the video camera unit 402.

The video camera unit 402 has a predetermined optical system, animage-taking device such as a CCD, and a signal processing unit foroutputting signals output from the image-taking device as digital videosignals, for example. These digital video signals are mapped to anHD-SDI format for example, and output from the video camera unit 402,for example. Of course, the digital video signals output from the videocamera unit 402 are not restricted to this example, and may be of otherformats as well.

The transmission unit 400 has a video signal encoding unit 410, digitalmodulation unit 411, and a wireless module unit 412. The video signalencoding unit 410 is configured basically in the same way as the imageencoding device 101 shown in FIG. 24.

At the transmission unit 400, the baseband digital video signals mappedto the HD-SDI format for example, and output. The digital video signalsare subjected to wavelet transformation and compression encoding byentropy encoding described above with reference to FIG. 25 at the videosignal encoding unit 410, so as to become a code stream which issupplied to the digital modulation unit 411. The digital modulation unit411 performs digital modulation of the supplied code stream into signalsof a format suitable for wireless communication, and outputs.

Also, digital audio signals and other signals, such as predeterminedcommands and data for example, are also supplied to the digitalmodulation unit 411. For example, the video camera unit 402 has amicrophone whereby collected sound is converted into audio signals, andfurther the audio signals are subjected to A/D conversion and output asdigital audio signals. Further, the video cameral unit 402 is capable ofoutputting certain commands and data. The commands and data may begenerated within the video camera unit 402, or an operation unit may beprovided to the video camera unit 402 with the commands and data beinggenerated in response to user operations made at the operating unit.Also, an arrangement may be made wherein an input device, for inputtingcommands and data, is connected to the video camera unit 402.

The digital modulation unit 411 performs digital modulation of thesedigital audio signals and other signals, and outputs. The digitalmodulated signals output from the digital modulation unit 411 aresupplied to the wireless module unit 412 and wirelessly transmitted froman antenna 413 as airwaves.

Upon receiving an ARQ (Auto Repeat Request) from the reception unit 401side, the wireless module unit 412 makes notification of this ARQ to thedigital modulation unit 411, so as to request a data resend.

The airwaves transmitted from the antenna 413 are received at an antenna420 of the reception device 401 side, and supplied to a wireless moduleunit 421. The reception device 401 has the wireless module unit 421front end unit 422, digital demodulation unit 423, and video signaldecoding unit 424. The video signal decoding unit 424 is basicallyconfigured the same way as with the image decoding unit 151 shown inFIG. 26.

The wireless module unit 421 supplies digital modulated signals based onthe received airwaves to the front end unit 422. The front end unit 422performs predetermined signal processing such as gain control to thesupplied digital modulated signals, for example, and supplies to thedigital demodulation unit 423. The digital demodulation unit 423demodulates the supplied digital modulated signals, and restores thecode stream.

The code stream restored at the digital demodulation unit 423 issupplied to the video signal decoding unit 424, the compressed encodingis decoded with the decoding method described above with reference toFIG. 27, and the baseband digital video signals are obtained. Thedecoded digital video signals are mapped to the HD-SDI format forexample, and output.

The digital demodulation unit 423 is also supplied with the digitalaudio signals and other signals subjected to digital modulation at thetransmission unit 400 side and transmitted. The digital demodulationunit 423 demodulates the signals wherein these digital audio signals andother signals have been subjected to digital modulation, and restoresand outputs the digital audio signals and other signals.

Also, the front end unit 422 performs error detection according to apredetermined method regarding the received signals supplied from thewireless module unit 421, and in the event that an error is detectedsuch as an erroneous frame having been received for example, outputs anARQ. The ARQ is supplied to the wireless module unit 421, andtransmitted form the antenna 420.

With such a configuration, the transmission unit 400 is built into arelatively small-sized video camera device having a video camera unit402 for example, a monitor device is connected to the reception device401, and the digital video signals output from the video signal decodingunit 424 are supplied to the monitor device. As long as the receptiondevice 401 is within the airwave range of the airwaves transmitted fromthe video camera device having the built-in transmission unit 400, thepictures taken with the video camera device can be watched on themonitor device with little delay, e.g., with a delay within one field orone frame.

Note that in the example shown in FIG. 29, communication between thetransmission unit 400 and the reception device 401 is performed usingwireless communication, so as to transmit video signals via wirelesscommunication, but this arrangement is not restricted to this example.For example, the transmission unit 400 and the reception device 401 maybe connected via a network such as the Internet. In this case, thewireless module unit 412 at the transmission unit 400 side and thewireless module unit 421 at the reception device side 401 side are eachcommunication interfaces capable of communication using IP (InternetProtocol).

Various applications can be conceived for the wireless transmissionsystem shown in FIG. 29. For example, this wireless transmission systemcan be applied to a videoconferencing system. An example of anarrangement would be to connect a simple video camera device capable ofUSB (Universal Serial Bus) connection to a computer device such as apersonal computer, with the computer device side implementing the videosignal encoding unit 410 and video signal decoding unit 424. The videosignal encoding unit 410 and video signal decoding unit 424 implementedat the computer device may be a hardware configuration, or may berealized by software running on the computer device.

For example, each of the members participating in the videoconferencewould be provided with a computer device and a video camera device to beconnected to the computer device, with the computer device beingconnected to a server device for providing the videoconference systemservice, by either cable or wireless network. Video signals output fromthe video camera device are supplied to the computer device via USBcable, and the encoding processing described above with reference toFIG. 25 is performed at the video signal encoding unit 410 within thecomputer device. The computer device transmits the code steam whereinthe videos signals have been encoded, to the server device or the like,via the network.

The server device transmits the received code stream to the computerdevice of each of the participating members, via the network. This codestream is received at the computer device of each of the participatingmembers, and is subjected to the decoding processing at the video signaldecoding unit 424 within the computer device described above withreference to FIG. 27. The image data output from the video signaldecoding unit 424 is displayed on the display unit of the computerdevice as a picture.

That is to say, video pictures taken by the video camera devices of theother participating members are displayed on the display units of thecomputer devices of each of the participating members. Accordingly,applying an embodiment of the present invention to the wirelesstransmission system means that the delay time from encoding videosignals taken with a video camera device to decoding thereof at thecomputer device of other participating members is short, so theunnatural sensation of the pictures of other participating members beingdisplayed on the display units of the computer devices of theparticipating members being delayed, can be reduced.

Further, an arrangement can be conceived wherein the video signalencoding unit 410 is installed at the video camera device side. Forexample, the transmission unit 400 is built into a video camera device.Such a configuration does away with the need for the video camera deviceto be connected to another device such as a computer device or the like.

Such a system mode up of the video camera device with the transmissionunit 400 built in, and the reception device 401, can be applied tovarious applications other than the above-described videoconferencingsystem. For example, as schematically shown in FIG. 30, this system canbe applied to a home gaming console. In FIG. 30, the transmission unit400 shown in FIG. 29 is built into a video camera device 500.

In the main unit 501 of the home gaming console, a bus for exampleconnects a CPU, RAM, ROM, a disk drive device compatible with CD-ROMs(Compact Disc Read Only Memory) and DVD-ROMs (Digital VersatileDisc-ROM), a graphics control unit for converting display controlsignals generated by the CPU into vide signals and outputting, an audioplayback unit for playing audio signals, and so forth, i.e., having aconfiguration generally like that of a computer device.

The main unit 501 of the home gaming console is controlled overall bythe CPU, following programs stored in the ROM beforehand, or programsrecorded in a CD-ROM or DVD-ROM mounted to the disk drive device. TheRAM is used as work memory for the CPU. The main unit 501 of the homegaming console has built in the reception device 401. digital videosignals output from the reception device 401, and other signals, aresupplied to the CPU via the bus, for example.

Let us say that with such a system, e.g., the main unit 501 of the homegaming console, software is running which can take images in the form ofdigital video signals supplied externally, as images within the game.For example, this game software is capable of using images in the formof digital video signals supplied externally as images within the game,and also recognize the movements of persons (players) within the image,and perform operations corresponding to the recognized motions.

The video camera device 500 encodes the shot digital video signals withthe encoding method described above with reference to FIG. 25 at thevideo signal encoding unit 410 within the built-in transmission unit400, modulates the code stream and the digital modulation unit 411 andsupplies to the wireless module unit 412, so s to be transmitted fromthe antenna 413. The transmitted airwaves are received at the antenna420 of the reception device 401 built into the main unit 501 of the homegaming console, the received signals being supplied to the digitaldemodulation unit 423 via the wireless module unit 421 and the front endunit 422.

The received signals are demodulated at the digital demodulation unit423 into a code stream, and supplied to the video signal decoding unit424. The video signal decoding unit 424 decodes the supplied code streamwith the decoding method described above with reference to FIG. 27, andoutputs the baseband digital video signals.

The baseband digital video signals output from the video signalsdecoding unit 424 are sent over the bus in the main unit 501 of the homegaming console, and temporarily stored in the RAM, for example. Upon thedigital video signals stored in the RAM being read out following apredetermined program, the CPU can detect movement of persons within theimage provided by the digital video signals, and use the image withinthe game.

Due to the delay time, from the images being shot with the video cameradevice 500 and the obtained digital video signals being encoded to thecode stream being decoded at the main unit 501 of the home gamingconsole and the images being obtained thereat, being short, responsivityof the game software running on the main unit 501 of the home gamingconsole as to the movement of the player improves, thereby improvingoperability of the game.

Note that such a video camera device 500 used with a home gaming consoleoften has a simple configuration due to restrictions on price, size, andso forth, and assumptions must be made that a CPU with high processingcapabilities and large-capacity memory may be unaffordable. Accordingly,using the encoding processing including the wavelet transformationprocessing according to an embodiment of the present invention allowsfor operation with a small memory capacity, since there is no need forlarge-capacity external memory. Also, an arrangement may be conceivedwherein wavelet transformation is performed at a low division level atthe video signal encoding unit 410 of the transmission unit 400 builtinto the video camera device 500. This further reduces the need formemory capacity.

Note that the video camera device 500 and the main unit 501 of the homegaming console have been described above as being connected by wirelesscommunication, but this arrangement is not restricted to this example.That is to say, the video camera device 500 and the main unit 501 of thehome gaming console may be connected by cable, via interfaces such asUSB, IEEE 1394, or the like.

The present invention has been described above by way of embodiments,whereby it is apparent that a wide range of applications thereof can bemade as long as belonging to a device or method for performing wavelettransformation of images or video signals, and a device or method forperforming wavelet inverse transformation for synthesizing filtering ofband-analyzed information so as to restore image or video signals.

That is to say, embodiments of the present invention are advantageouslyapplied to devices or systems, wherein image signals or images of videosignals are compressed, transmitted received, decompressed, and output,as described above with reference to FIGS. 28 through 30, by providingan encoding arrangement is provided downstream of the wavelettransformation, as with the image encoding device 101 shown in FIG. 24.Embodiments of the present invention are particularly advantageous withdevices or systems wherein short delay from compression encoding todecoding and output of images is demanded.

Another application is remote medical diagnosis and treatment usingremotely operable instruments or devices, while viewing images takenwith a video camera, for example.

Another application is compression encoding and transmission of digitalvideo signals, and decoding of digital video signals subjected tocompression encoding, in systems such as used in broadcasting stationsand the like.

Another application is to systems for distributing video of livecoverage.

Another application is to remote educational systems, wherein studentsand teachers can communicate interactively.

Further applications include, but are not restricted to, systems fortransmitting image data taken with mobile terminals having image-takingfunctions, such as cellular phones with camera functions;videoconferencing systems; surveillance systems for recording imagestaken with a monitoring camera with a recorder; wireless imagetransmission systems; interactive gaming applications; and so forth.

The series of processing in these various applications can be realizedby hardware or by software, as with the case illustrated in FIG. 30.

In the case of realizing the series of processing by software, a programmaking up the software is installed in a computer which has dedicatedhardware built in, or installed in a general-purpose computer forexample, capable of executing various functions by various types ofprograms being installed therein, from a program recording medium.

FIG. 31 is a block diagram illustrating an example of the configurationof a personal computer 701 for executing the above-described series ofprocessing with a software program. A CPU 711 executes various types ofprograms according to programs stored in ROM 712 or a storage unit 718.RAM 713 stores programs and data used by the CPU 711 as necessary. TheCPU 711, ROM 712, and RAM 713 are mutually connected by a bus 714.

An input/output interface 715 is also connected to the CPU 711 via thebus 714. Connected to the input/output interface 715 are an input unit716 including a keyboard mouse, microphone, and so forth, and an outputunit 717 including a display, speaker, and so forth. The CPU 711executes various types of processing in response to commands input formthe input unit 716. The CPU 711 outputs the results of processing to theoutput unit 717.

The storage unit 718 connected to the input/output interface 715 isconfigured of a hard disk for example, and stores various types ofprograms and data which the CPU 711 executes or uses. A communicationunit 719 communicates with external devices via a network such as theInternet, a Local Area Network, or the like.

Programs may also be obtained via the communication unit 719 and storedin the storage unit 718.

A drive 720 connected to the input/output interface 715 drives aremovable media 721 mounted thereto, such as a magnetic disk opticaldisk magneto-optical disk, semiconductor memory, or the like, so as toobtain programs or data recorded therein. The programs and data obtainedare transferred to the storage unit 718 as necessary and stored.

A program recording medium for storing programs which are installed inthe computer in a computer-executable form includes the removable media721 shown in FIG. 31 which is packaged media including magnetic disks(including flexible disks), optical disks (including CD-ROM and DVD),magneto-optical disks, semiconductor memory, etc., the ROM 712 whereprograms or temporarily or permanently stored, a hard disk making up thestorage unit 718, and so forth. Storing of programs to the programrecording medium can also be performed using wire or wirelesscommunication media such as a Local Area Network, the Internet, digitalsatellite broadcast, etc., via the communication unit 719 serving as aninterface with a router, modem, etc., as necessary.

While the steps describing the program stored in the program recordingmedium in the present Specification may of course be performed in thetime-sequence described of course, but is not restricted to thistime-sequence, and may be executed in parallel, or individually.

Further, the term “system” as used in the present Specification refersto the entirety of equipment configured of multiple devices.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. A wavelet transformation device for performing wavelet transformationat a plurality of division levels as to image signals, the wavelettransformation device comprising: horizontal filtering means forsubjecting the image signals to horizontal direction lowband analysisfiltering and highband analysis filtering; buffering means comprising aplurality of buffers which are independent for each of the divisionlevels, for holding frequency components, which are generated as theresults of the horizontal direction analysis filtering by the horizontalfiltering means, for each of the division levels; and interleaving meanscomprising at least one interleaving unit for interleaving brightnesscomponents and color difference components of the image signals and forinterleaving highband components and lowband components of bufferedfilter results.
 2. The wavelet transformation device according to claim1, wherein the lowband components and highband components in thefrequency components obtained as the results of the horizontal directionanalysis filtering are reordered and held in the buffer.
 3. The wavelettransformation device according to claim 1, further comprising:reordering means for performing reordering of brightness signals andcolor difference signals, which are elements of the image signals,wherein the horizontal filtering means subject the image signalsreordered by the reordering means to horizontal direction lowbandanalysis filtering and highband analysis filtering.
 4. The wavelettransformation device according to claim 3, wherein the frequencycomponents of the brightness signals and the frequency components of thecolor difference signals generated as a result of the horizontalanalysis filtering performed by the horizontal filtering means are eachheld separately in the buffers.
 5. The wavelet transformation deviceaccording to claim 1, further comprising: vertical filtering means forsubjecting the frequency components, generated as a result of thehorizontal direction analysis filtering, that are held in the buffers,to vertical direction lowband analysis filtering and highband analysisfiltering.
 6. The wavelet transformation device according to claim 5,the vertical filtering means further comprising: brightness signalvertical filtering means for subjecting frequency components ofbrightness signals, which are elements of the image signals, to verticaldirection lowband analysis filtering and highband analysis filtering;and color difference signal vertical filtering means for subjectingfrequency components of color difference signals, which are elements ofthe image signals, to vertical direction lowband analysis filtering andhighband analysis filtering.
 7. The wavelet transformation deviceaccording to claim 6, wherein the brightness signal vertical filteringmeans and the color difference signal vertical filtering means areoperated in parallel.
 8. The wavelet transformation device according toclaim 7, further comprising: reordering means for performing reorderingof the lowband components of the frequency components of the brightnesssignals generated as a result of the vertical direction analysisfiltering performed by the brightness signal vertical filtering means,and the frequency components of the color difference signals generatedas a result of the vertical direction analysis filtering performed bythe color difference signal vertical filtering means; wherein thehorizontal filtering means subject the lowband components reordered bythe reordering means to horizontal direction lowband analysis filteringand highband analysis filtering.
 9. The wavelet transformation deviceaccording to claim 8, wherein prior to reordering by the reorderingmeans, one of the brightness signal vertical filtering means and thecolor difference signal vertical filtering means stands by until theanalysis filtering of the other ends.
 10. The wavelet transformationdevice according to 8, wherein the horizontal filtering means performthe horizontal direction lowband analysis filtering and highbandanalysis filtering to a predetermined number of levels.
 11. The wavelettransformation device according to claim 8, wherein the frequencycomponents of the brightness signals and the frequency components of thecolor difference signals generated as a result of the horizontaldirection analysis filtering performed by the horizontal filtering meansare separately held in the buffers.
 12. The wavelet transformationdevice according to claim 5, wherein the horizontal filtering means andthe vertical filtering means perform analysis filtering on the lowestband frequency components in a hierarchical manner.
 13. The wavelettransformation device according to claim 5, wherein the horizontalfiltering means and the vertical filtering means are realized by alifting scheme of the wavelet transformation.
 14. The wavelettransformation device according to claim 5, wherein the horizontalfiltering means input the image signals in increments of lines, andperform the horizontal direction lowband analysis filtering and highbandanalysis filtering each time the number of samples in the horizontaldirection reaches a predetermined number, and wherein the verticalfiltering means perform the vertical direction lowband analysisfiltering and highband analysis filtering each time the number of linesin the vertical direction of the frequency component in the results ofthe horizontal direction analysis filtering performed by the horizontalfiltering means reach a predetermined number.
 15. The wavelettransformation device according to claim 5, wherein the image signalsare video signals comprising a plurality of pictures, wherein thewavelet transformation device further comprises detecting means fordetecting the end of each picture by detecting vertical synchronizationsignals of the video signals, and wherein the horizontal filtering meansand the vertical filtering means perform analysis filtering for eachpicture.
 16. A wavelet transformation method of a wavelet transformationdevice for performing wavelet transformation at a plurality of divisionlevels as to image signals, the method comprising: subjecting the imagesignals to horizontal direction lowband analysis filtering and highbandanalysis filtering; holding frequency components, which are generated asthe results of the horizontal direction analysis filtering, for each ofthe division levels, in a plurality of buffers which are independent foreach of the division levels; interleaving brightness components andcolor difference components of the image signals; and interleavinghighband components and lowband components of buffered filter results.17. A wavelet transformation device for performing wavelettransformation at a plurality of division levels as to image signals,the wavelet transformation device comprising: a horizontal filteringunit for subjecting the image signals to horizontal direction lowbandanalysis filtering and highband analysis filtering; a plurality ofbuffers which are independent for each of the division levels, forholding frequency components, which are generated as the results of thehorizontal direction analysis filtering by the horizontal analysisfiltering unit, for each of the division levels; and at least oneinterleaving unit for interleaving brightness components and colordifference components of the image signals and for interleaving highbandcomponents and lowband components of buffered filter results.
 18. Awavelet inverse transformation device for performing wavelet inversetransformation as to frequency components, generated by a plurality ofdivision levels of wavelet transformations having been performed as toimage signals, thereby reconstructing an image, the wavelet inversetransformation device comprising: horizontal filtering means forsubjecting the frequency components to horizontal direction lowbandsynthesizing filtering and highband synthesizing filtering; bufferingmeans comprising a plurality of buffers which are independent for eachof the division levels except for the lowest band, for holding frequencycomponents, which are generated as the results of the horizontaldirection synthesizing filtering by the horizontal filtering means, foreach of the division levels; and interleaving means comprising at leastone interleaving unit for interleaving brightness components and colordifference components of the image signals and for interleaving highbandcomponents and lowband components of buffered filter results.
 19. Thewavelet inverse transformation device according to claim 18, furthercomprising: vertical filtering means for subjecting the frequencycomponents to vertical direction lowband analysis filtering and highbandanalysis filtering, wherein the horizontal filtering means subject thefrequency components generated as a result of the vertical directionsynthesizing filtering to the horizontal direction lowband synthesizingfiltering and highband synthesizing filtering.
 20. The wavelet inversetransformation device according to claim 19, wherein the verticalfiltering means and the horizontal filtering means are realized by alifting scheme of the wavelet inverse transformation.
 21. The waveletinverse transformation device according to claim 19, wherein thehorizontal filtering means input the frequency components in incrementsof lines, and perform the horizontal direction lowband synthesizingfiltering and highband synthesizing filtering each time the number ofsamples in the horizontal direction reaches a predetermined number, andwherein the vertical filtering means perform the vertical directionlowband synthesizing filtering and highband synthesizing filtering eachtime the number of lines in the vertical direction of the frequencycomponent in the results of the horizontal direction synthesizingfiltering performed by the horizontal filtering means reach apredetermined number.
 22. The wavelet inverse transformation deviceaccording to claim 19, wherein the image signals are video signalscomprising a plurality of pictures, divided into a plurality offrequency components by performing analysis filtering on the lowest bandfrequency components in a hierarchical manner, and wherein the verticalfiltering means and the horizontal filtering means perform synthesizingfiltering in a hierarchical manner from, of a plurality of frequencycomponents, a predetermined number of frequency components including thelowest band frequency components, ultimately generating a picture. 23.The wavelet inverse transformation device according to claim 22, furthercomprising: vertical synchronizing signal insertion means for insertingvertical synchronizing signals between the pictures generated by thevertical filtering means and the horizontal filtering means, therebygenerating video signals.
 24. The wavelet inverse transformation deviceaccording to claim 19, wherein the vertical filtering means, thehorizontal filtering means, and the buffers, are provided separately forbrightness signals and for color difference signals, which are elementsof the image signals, and wherein the vertical filtering means, thehorizontal filtering means, and the buffers, for brightness signals, andthe vertical filtering means, the horizontal filtering means, and thebuffers, for color difference signals, are operated in parallel.
 25. Awavelet inverse transformation method for performing wavelet inversetransformation as to frequency components generated by a plurality ofdivision levels of wavelet transformations being performed as to imagesignals, thereby reconstructing an image, the method comprising:subjecting the frequency components to horizontal direction lowbandsynthesizing filtering and highband synthesizing filtering; holdingfrequency components, which are generated as the results of thehorizontal direction synthesizing filtering by the horizontal filteringmeans, for each of the division levels, in a plurality of buffers whichare independent for each of the division levels except for the lowestband; interleaving brightness components and color difference componentsof the image signals; and interleaving highband components and lowbandcomponents of buffered filter results.
 26. A wavelet inversetransformation device for performing wavelet inverse transformation asto frequency components, generated by a plurality of division levels ofwavelet transformations having been performed as to image signals,thereby reconstructing an image, the wavelet inverse transformationdevice comprising: a horizontal filtering unit for subjecting thefrequency components to horizontal direction lowband synthesizingfiltering and highband synthesizing filtering; a plurality of bufferswhich are independent for each of the division levels except for thelowest band, for holding frequency components, which are generated asthe results of the horizontal direction synthesizing filtering by thehorizontal filtering unit, for each of the division levels; and at leastone interleaving unit for interleaving brightness components and colordifference components of the image signals and for interleaving highbandcomponents and lowband components of buffered filter results.